An intent recognition method, apparatus, device, and storage medium

Abstract

This invention discloses an intent recognition method, apparatus, device, and storage medium. The method includes: acquiring text to be recognized; inputting the text to be recognized into an intent recognition model to obtain a target intent, wherein the intent recognition model is obtained by iterating a first model using a target sample set, the target sample set including: text samples, each text sample carrying a role label, a predicate sentiment label, and a score corresponding to at least one intent label. Through the technical solution of this invention, the role and predicate sentiment information affecting intent in the recognized text can be effectively integrated into the semantic representation, improving the expressive ability of customer questions.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to an intent recognition method, apparatus, device and storage medium. Background Technology

[0002] Intelligent dialogue is an important application in the field of natural language processing (NLP), with numerous successful cases in industries such as finance, telecommunications, and e-commerce. Dialogue systems can help businesses quickly, accurately, and efficiently resolve common, frequently asked customer questions, thereby saving them significant customer service manpower costs.

[0003] Currently, intent recognition is mainly achieved through text classification. Various feature extractors (CNN, RNN, and transformer, etc.) are used to semantically represent customer queries, thereby mapping them to multiple intents.

[0004] However, in practical application, it has been found that customers' intentions are often quite subjective, and it is difficult to accurately identify their intentions using only the semantic representation of the customer's question. On the other hand, common words in spoken language (such as "you," "I," "is," and "have") have relatively simple meanings and are even considered stop words in many cases, yet they express completely different intentions in the customer's questions. Summary of the Invention

[0005] This invention provides an intent recognition method, apparatus, device, and storage medium that can effectively integrate the roles and predicate sentiment information that influence intent in the identified text into semantic representation, thereby improving the expressive ability of customer questions.

[0006] According to one aspect of the present invention, an intent recognition method is provided, comprising:

[0007] Obtain the text to be recognized;

[0008] The text to be identified is input into the intent recognition model to obtain the target intent. The intent recognition model is obtained by iterating the first model through a target sample set. The target sample set includes: text samples, which carry role tags, predicate sentiment tags and scores corresponding to at least one intent tag.

[0009] Optionally, the first model can be iterated using the target sample set, including:

[0010] Establish the first model;

[0011] The text sample is input into the first model to obtain the predicted role label, predicted predicate sentiment, and scores corresponding to at least one predicted intent;

[0012] The parameters of the first model are trained based on the objective function formed by the predicted role label, predicted predicate sentiment, score corresponding to at least one predicted intent, role label, predicate sentiment label and score corresponding to at least one intent label carried by the text sample;

[0013] Return to the operation of inputting the text sample into the first model to obtain the predicted role label, predicted predicate sentiment, and at least one score corresponding to the predicted intent, until the intent recognition model is obtained.

[0014] Optionally, the text to be recognized is input into an intent recognition model to obtain the target intent, including:

[0015] The text to be identified is encoded using BERT to obtain the semantic information vector corresponding to each character in the text to be identified and the initial semantic information vector corresponding to the text to be identified.

[0016] The semantic information vector corresponding to each character in the text to be identified is input into the label determination layer to obtain the first label sequence corresponding to the text to be identified.

[0017] An attention vector is generated based on the first label sequence, the semantic information vector corresponding to each character in the text to be identified, and the initial semantic information corresponding to the text to be identified.

[0018] The attention vector and the initial semantic information vector corresponding to the text to be identified are concatenated to obtain the target fusion vector;

[0019] The target fusion vector is input into the activation layer to obtain the target intent.

[0020] Optionally, an attention vector is generated based on the first label sequence, the semantic information vector corresponding to each character in the text to be identified, and the initial semantic information corresponding to the text to be identified, including:

[0021] Extract target characters from the text to be identified based on the first tag sequence;

[0022] Obtain the semantic information vector corresponding to the target character;

[0023] The weight of each character in the text to be identified is determined based on the semantic information vector corresponding to the target character and the initial semantic information vector corresponding to the text to be identified.

[0024] The attention vector is obtained by weighted summation of the semantic information vectors corresponding to the target characters.

[0025] Optionally, the weight of each character in the text to be identified is determined based on the semantic information vector corresponding to the target character and the initial semantic information vector corresponding to the text to be identified, including:

[0026] The initial semantic information vector corresponding to the text to be identified and the semantic information vector corresponding to the target character are multiplied by a dot to obtain the score for each character.

[0027] The score corresponding to each character is normalized to obtain the weight of each character.

[0028] Optionally, the vector corresponding to each character in the text to be identified is input into the label determination layer to obtain the first label sequence corresponding to the text to be identified, including:

[0029] The vector corresponding to each character in the text to be identified is input into the CRF layer to obtain the score of at least one label sequence corresponding to the text to be identified;

[0030] The highest-scoring tag sequence among at least one tag sequence corresponding to the text to be identified is determined as the first tag sequence.

[0031] According to another aspect of the present invention, an intent recognition device is provided, comprising:

[0032] The acquisition module is used to acquire the text to be recognized;

[0033] The model input module is used to input the text to be recognized into the intent recognition model to obtain the target intent. The intent recognition model is obtained by iterating the first model through a target sample set. The target sample set includes: text samples, which carry role labels, predicate sentiment labels and scores corresponding to at least one intent label.

[0034] Optional, model input module, specifically used for:

[0035] Establish the first model;

[0036] The text sample is input into the first model to obtain the predicted role label, predicted predicate sentiment, and scores corresponding to at least one predicted intent;

[0037] The parameters of the first model are trained based on the objective function formed by the predicted role label, predicted predicate sentiment, score corresponding to at least one predicted intent, role label, predicate sentiment label and score corresponding to at least one intent label carried by the text sample;

[0038] Return to the operation of inputting the text sample into the first model to obtain the predicted role label, predicted predicate sentiment, and at least one score corresponding to the predicted intent, until the intent recognition model is obtained.

[0039] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0040] At least one processor; and

[0041] A memory communicatively connected to the at least one processor; wherein,

[0042] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the intent recognition method according to any embodiment of the present invention.

[0043] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the intent recognition method according to any embodiment of the present invention.

[0044] This invention provides an embodiment of the invention that obtains text to be identified and inputs it into an intent recognition model to obtain a target intent. The intent recognition model is obtained by iterating a first model with a target sample set. The target sample set includes text samples, each carrying a role label, a predicate sentiment label, and a score corresponding to at least one intent label. Through the technical solution of this invention, the role and predicate sentiment information that influence intent in the identified text can be effectively integrated into the semantic representation, thereby improving the expressive ability of customer questions.

[0045] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of an intent recognition method provided according to Embodiment 1 of the present invention;

[0048] Figure 2 This is an example flowchart of an intent recognition method provided in Embodiment 1 of the present invention;

[0049] Figure 3This is a schematic diagram of the structure of an intent recognition device according to Embodiment 2 of the present invention;

[0050] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to Embodiment 3 of the present invention. Detailed Implementation

[0051] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0052] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0053] Example 1

[0054] Figure 1 This is a flowchart of an intent recognition method according to Embodiment 1 of the present invention. This embodiment is applicable to the situation of recognizing customer intent in intelligent dialogue. The method can be executed by the intent recognition device in this embodiment of the invention, which can be implemented in software and / or hardware, such as... Figure 1 As shown, the method specifically includes the following steps:

[0055] S110, Obtain the text to be recognized.

[0056] The text to be recognized can be understood as the text identified from the user's questions in the intelligent dialogue. The text to be recognized can be obtained through text recognition methods, and this embodiment of the invention does not limit this. The text to be recognized may include multiple characters. In this embodiment, the text to be recognized can be segmented or otherwise processed to obtain a sequence representing each character. For example, the input sequence is... , where x0…x TThis can be represented by the individual characters of the input text, where T is the length of the input sequence. In this embodiment, the text to be recognized can be obtained.

[0057] S120, the text to be identified is input into the intent recognition model to obtain the target intent, wherein the intent recognition model is obtained by iterating the first model through a target sample set, and the target sample set includes: text samples, the text samples carrying role tags, predicate sentiment tags and scores corresponding to at least one intent tag.

[0058] The target intent reflects the true intent of the identified text. The target intent can be obtained by inputting the text to be identified into an intent recognition model. The intent recognition model can be used to identify the intent in text information. The intent recognition model can be a pre-trained model. The intent recognition model can be obtained by iterating the first model using a target sample set. Target samples can include text samples, which can carry role tags, predicate sentiment tags, and scores corresponding to at least one intent tag. Role tags can be understood as various tags representing roles. Predicate sentiment tags can be understood as various tags representing predicate sentiment; predicate sentiment tags can include positive and negative sentiment tags. Intent tags can be understood as tags representing customer intent. In this embodiment, each character in the identified text has a corresponding tag. In this embodiment, the score corresponding to each tag can be different. In this embodiment, the number of scores corresponding to each tag can be pre-set. The text samples in this embodiment can carry role tags, predicate sentiment tags, and scores corresponding to at least one intent tag.

[0059] For example, the tags carried by the text in this embodiment may include SR (single word role), BR (role start), IR (role middle), S-POS (single positive predicate), S-NEG (single negative predicate), S-NEU (single neutral predicate), B-POS (positive predicate start), I-POS (positive predicate middle), B-NEG (negative predicate start), I-NEG (negative predicate middle), B-NEU (neutral predicate start), I-NEU (neutral predicate middle), and O (other) tags.

[0060] In this embodiment, the text input intent recognition model can be used to obtain the target intent.

[0061] In this embodiment, iterating the first model using a target sample set includes: establishing a first model; inputting the text sample into the first model to obtain a predicted role label, a predicted predicate sentiment, and a score corresponding to at least one predicted intent; training the parameters of the first model based on a target function formed by the predicted role label, the predicted predicate sentiment, the score corresponding to at least one predicted intent, the role label carried by the text sample, the predicate sentiment label, and the score corresponding to at least one intent label; and returning to execute the operation of inputting the text sample into the first model to obtain a predicted role label, a predicted predicate sentiment, and a score corresponding to at least one predicted intent, until an intent recognition model is obtained.

[0062] In this embodiment, the first model can be understood as the first model for intent recognition. The first model can be used for iterative training using a target sample set to obtain the intent recognition model. Predicting the role label can be obtained by inputting text samples into the first model. Predicting the predicate sentiment can be obtained by inputting text samples into the first model. The score corresponding to the predicted intent can be obtained by inputting text samples into the first model. There can be at least one score corresponding to the predicted intent, or multiple scores. The objective function can characterize the loss function representing the difference between the scores corresponding to the predicted role label, predicted predicate sentiment, and at least one predicted intent, and the scores corresponding to the role label, predicate sentiment label, and at least one intent label carried by the text sample. The objective function can be used to train the parameters of the first model. In this embodiment, the operation of inputting text samples into the first model to obtain the predicted role label, predicted predicate sentiment, and at least one predicted intent score can be performed iteratively until the intent recognition model is obtained.

[0063] For example, the intent recognition model is trained end-to-end via backpropagation. In this embodiment, cross-entropy can be used as the loss function, which is also the objective function loss:

[0064] (1);

[0065] in, This represents a sample in the training text sample dataset D. Indicates the first The first text sample The actual score of an intention, Indicates the first The first text sample The predicted score for each intention, These are the training parameters of the model. It is the L2 regularization coefficient.

[0066] In this embodiment, the training and decoding process of the character expression and predicate sentiment label recognition model can be implemented using a dynamically programmed Viterbi search algorithm. The Viterbi algorithm is a dynamic programming algorithm used to find the Viterbi path-hidden state sequence most likely to generate the sequence of observed events.

[0067] In this embodiment, a first model can be established; text samples are input into the first model to obtain predicted role labels, predicted predicate sentiment, and scores corresponding to at least one predicted intent; the parameters of the first model are trained based on an objective function formed by the predicted role labels, predicted predicate sentiment, scores corresponding to at least one predicted intent, role labels carried by the text sample, predicate sentiment labels, and scores corresponding to at least one intent label; the process of inputting text samples into the first model to obtain predicted role labels, predicted predicate sentiment, and scores corresponding to at least one predicted intent is repeated until an intent recognition model is obtained. Through this setup, the first model can be trained multiple times using the objective function to obtain the intent recognition model, further improving its accuracy. This model reduces the error accumulation caused by pipelined recognition of predicates and sentiments, while simultaneously extracting role and predicate sentiment information from the question.

[0068] In this embodiment, inputting the text to be recognized into an intent recognition model to obtain the target intent includes: performing BERT encoding on the text to be recognized to obtain a semantic information vector corresponding to each character in the text to be recognized and an initial semantic information vector corresponding to the text to be recognized; inputting the semantic information vector corresponding to each character in the text to be recognized into a label determination layer to obtain a first label sequence corresponding to the text to be recognized; generating an attention vector based on the first label sequence, the semantic information vector corresponding to each character in the text to be recognized, and the initial semantic information vector corresponding to the text to be recognized; concatenating the attention vector and the initial semantic information vector corresponding to the text to be recognized to obtain a target fusion vector; and inputting the target fusion vector into an activation layer to obtain the target intent.

[0069] BERT encoding can be understood as encoding the text to be recognized into a format that the model's computer language can recognize. In this embodiment, BERT encoding can be performed on the text to be recognized to obtain the semantic information vector corresponding to each character in the text and the initial semantic information vector corresponding to the text. The label determination layer can be a linear chain CRF layer (CRF, conditional random fields). The first label sequence can be obtained by inputting the semantic information vector corresponding to each character in the text to be recognized into the label determination layer. The attention vector can represent the total attention to the role and predicate sentiment information from the perspective of the entire input sequence. The attention vector can be generated based on the first label sequence, the semantic information vector corresponding to each character in the text to be recognized, and the initial semantic information of the text to be recognized. The target fusion vector can be obtained by concatenating the attention vector and the initial semantic information vector corresponding to the text to be recognized. The target intent can be obtained by inputting the target fusion vector into the activation layer. In this embodiment, the activation layer can be a softmax activation function layer.

[0070] For example, the overall flowchart in this embodiment is shown below. Figure 2 As shown. In this embodiment, a customer question input sequence is given. Where T is the length of the input sequence. The model outputs not only auxiliary semantic roles and predicate sentiment sequence labels. ,in, ,gather Possible label values ​​are: SR, BR, IR, S-POS, S-NEG, S-NEU, B-POS, I-POS, B-NEG, I-NEG, B-NEU, I-NEU, and O, which can represent: single word role, role start, role middle, single positive predicate, single negative predicate, single neutral predicate, positive predicate start, positive predicate middle, negative predicate start, negative predicate middle, neutral predicate start, neutral predicate middle, and others, respectively. Furthermore, the intent recognition model in this embodiment primarily outputs intent recognition classification labels. Where L represents the total number of intent labels. Specifically, in this embodiment, the input text sequence x of the customer's question to be recognized is encoded by BERT and then fed into a conditional random field (CRF) layer, outputting a label for each character in the sequence. The sequence label contains both role information from the customer's question (distinguished by SR, BR, and IR) and predicate sentiment information from the customer's question (distinguished by S-{POS, NEG, NEU}, B-{POS, NEG, NEU}, and I-{POS, NEG, NEU}). Based on the first sequence label... The system filters key role and predicate sentiment information from the input sequence and fuses it with the semantic information of the entire sequence using an attention mechanism. It's important to note that this attention mechanism is similar to a gating mechanism, aiming to incorporate information more relevant to the current intention's positive sentiment into the semantic representation. Finally, the semantic information of the entire sequence and the fused information are concatenated to obtain the target intention label. This embodiment utilizes a role and predicate emotion recognition task. and intent recognition task Multi-task joint training, with intent recognition as the primary task and role and predicate emotion recognition as auxiliary tasks, improves the end-to-end intent recognition performance through backpropagation.

[0071] Specifically, the input sequence of the text to be input. After BERT encoding, the result is... ,in, It is the first in the sequence The vectorized representation of each character, and the first vector [CLS] represents the semantic representation of the entire input sequence. [CLS] can be understood as a placeholder character. This embodiment uses an end-to-end predicate sentiment model, rather than a pipeline approach (i.e., identifying predicate labels first, then sentiment labels). Furthermore, role recognition and predicate recognition are treated as sequence labeling tasks, and the labeling method follows the BIOS model (BIOS, begin-inside-or-other-sequence). For identified role sequences, they are distinguished using SR or BR, IR. For example, the sequence label for "I" in "I don't want to redeem, I want to cancel the order" is SR. For identified predicates and their corresponding positive sentiments, they are distinguished using S-POS or B-POS, I-POS. For example, the predicate "redeem" in "I don't want to redeem, I want to cancel the order" has a negative sentiment, so the corresponding sequence labels are B-NEG and I-NEG; while the predicate "cancel the order" has a positive sentiment, so the corresponding sequence labels are B-POS and I-POS.

[0072] In this embodiment, the text to be recognized can be BERT encoded to obtain a semantic information vector corresponding to each character in the text and an initial semantic information vector corresponding to the text. The semantic information vector corresponding to each character in the text is then input into a label determination layer to obtain a first label sequence corresponding to the text. An attention vector is generated based on the first label sequence, the semantic information vector corresponding to each character in the text, and the initial semantic information vector corresponding to the text. The attention vector and the initial semantic information vector corresponding to the text are concatenated to obtain a target fusion vector. The target fusion vector is then input into an activation layer to obtain the predicted intent. This embodiment, through this setup, can effectively fuse the main task of intent recognition with the auxiliary tasks of role and predicate sentiment recognition based on multi-task joint learning, thereby improving the intent recognition effect.

[0073] In this embodiment, generating an attention vector based on the first label sequence, the semantic information vector corresponding to each character in the text to be identified, and the initial semantic information corresponding to the text to be identified includes: extracting target characters from the text to be identified based on the first label sequence; obtaining the semantic information vector corresponding to the target character; determining the weight corresponding to each character in the text to be identified based on the semantic information vector corresponding to the target character and the initial semantic information vector corresponding to the text to be identified; and performing a weighted summation of the semantic information vector corresponding to the target character based on the weights to obtain the attention vector.

[0074] The target character can be extracted from the text to be recognized based on the first label sequence. The target character can be a character from a role label or a predicate sentiment label. The weight corresponding to each character in the text to be recognized can be determined based on the semantic information vector corresponding to the target character and the initial semantic information vector corresponding to the text to be recognized. The attention vector can be obtained by weighted summation of the semantic information vector corresponding to the target character based on the weights.

[0075] In this embodiment, target characters can be extracted from the text to be recognized based on the first label sequence; the semantic information vector corresponding to the target character can be obtained; the weight of each character in the text to be recognized can be determined according to the semantic information vector corresponding to the target character and the initial semantic information vector corresponding to the text to be recognized; and the semantic information vector corresponding to the target character can be weighted and summed based on the weight to obtain the attention vector. Through this setup, the semantic, role, and predicate sentiment interaction methods based on the attention mechanism can be used to enable the intent recognition model to effectively integrate the role and predicate sentiment information that influences the intent in the question into the semantic representation, thereby improving the expressive ability of customer questions.

[0076] In this embodiment, determining the weight of each character in the text to be identified based on the semantic information vector corresponding to the target character and the initial semantic information vector corresponding to the text to be identified includes: performing a dot product between the initial semantic information vector corresponding to the text to be identified and the semantic information vector corresponding to the target character to obtain a score for each character; and normalizing the score for each character to obtain the weight for each character.

[0077] In this embodiment, the initial semantic information vector corresponding to the text to be recognized and the semantic information vector corresponding to the target character are multiplied by a dot to obtain a score for each character. Then, the score for each character is normalized to obtain the weight for each character. This setup in this embodiment, by determining the weight for each character in the text to be recognized, facilitates the subsequent acquisition of the attention vector.

[0078] For example, in this embodiment, role information and predicate sentiment information can be integrated into semantic information through an attention mechanism to achieve intent recognition. Specifically, this embodiment can be based on BERT for intent recognition, typically by directly using the vector corresponding to [CLS]. That is, the semantic representation vector of the entire input sequence. Based on this, the first sequence, combined with the role and predicate sentiment sequence annotation results That is, if The first in If each element is a character or predicate sentiment tag, then its corresponding vector is used. Otherwise, use the zero vector. The core information is obtained as shown in the following formula. :

[0079] (2);

[0080] in, It is the first in the sequence. Vectorized representation of a character; It is a zero vector, representing non-role and predicate sentiment information. As a query, for Perform attention calculation, i.e. and any vector in Perform a dot product operation to obtain a fraction. Then for T Performing a softmax operation yields the normalized scores, which are the weights. , weight With the corresponding vector Multiplying these results yields the level of attention given to the emotional information of that role or predicate. Summing these results gives the attention vector. This represents the overall level of attention paid to the emotional information of the characters and predicates from the perspective of the entire input sequence. The detailed calculation process of the attention vector is shown in equations (3)-(5):

[0081] (3);

[0082] (4);

[0083] (5);

[0084] in, These are the parameters that need to be learned. Then, the semantic vector of the entire sequence... With attention vector The vector is obtained by concatenating the two vectors. Based on the fused vector, the target intent is output through the softmax activation function.

[0085] (6);

[0086] in, These are training parameters. It is a weight matrix. It is a bias. As can be seen from equation (6), role and predicate sentiment information are added in this embodiment. In extreme cases, if the customer's question does not mention any role or predicate sentiment, then, after attention calculation, the corresponding vector is obtained. This is the zero vector. At this point, it is equivalent to having only the semantic vector of the entire sequence. It plays a role in predicting the final intent, thereby achieving the goal of identifying the intent behind customer questions.

[0087] In this embodiment, the vector corresponding to each character in the text to be identified is input into the label determination layer to obtain the first label sequence corresponding to the text to be identified, including: inputting the vector corresponding to each character in the text to be identified into the CRF layer to obtain the score of at least one label sequence corresponding to the text to be identified; and determining the label sequence with the highest score among the at least one label sequences corresponding to the text to be identified as the first label sequence.

[0088] The first label sequence can be the label sequence with the highest score among at least one label sequence corresponding to the text to be identified.

[0089] Specifically, in this embodiment, based on the BERT output layer, the vector corresponding to each character in the text to be recognized is input into the linear chain CRF layer, and the sequence-level score is calculated as follows:

[0090] (7);

[0091] in, This indicates that the output sequence of the input sequence x is the first label sequence. The score consists of two parts: the emission probability and the transition probability. The label that can represent the t-th character is probability of occurrence It can represent the label Transfer to label The transition probability. It is worth noting that the emission matrix... h is obtained through a linear transformation, the transition matrix. It is randomly initialized and updated during training. Therefore, the sequence label probabilities for role recognition and predicate sentiment recognition can be calculated as follows:

[0092] (8);

[0093] In this embodiment, the vector corresponding to each character in the text to be recognized can be input into the CRF layer to obtain the score of at least one label sequence corresponding to the text to be recognized; then, the label sequence with the highest score among the at least one label sequences corresponding to the text to be recognized is determined as the first label sequence. With this setting, a linear chain CRF layer can be used in this embodiment, thereby finding the globally optimal label sequence.

[0094] This invention provides an embodiment of the invention that obtains text to be identified and inputs it into an intent recognition model to obtain a target intent. The intent recognition model is obtained by iterating a first model with a target sample set. The target sample set includes text samples, each carrying a role label, a predicate sentiment label, and a score corresponding to at least one intent label. Through the technical solution of this invention, the role and predicate sentiment information that influence intent in the identified text can be effectively integrated into the semantic representation, thereby improving the expressive ability of customer questions.

[0095] Example 2

[0096] Figure 3 This is a schematic diagram of an intent recognition device according to Embodiment 2 of the present invention. This embodiment is applicable to situations where customer intent is recognized in intelligent dialogue. The device can be implemented using software and / or hardware, and can be integrated into any device that provides data processing capabilities, such as… Figure 3 As shown, the intent recognition device specifically includes: an acquisition module 310 and a model input module 320.

[0097] The acquisition module 310 is used to acquire the text to be recognized;

[0098] The model input module 320 is used to input the text to be recognized into the intent recognition model to obtain the target intent. The intent recognition model is obtained by iterating the first model through a target sample set. The target sample set includes: text samples, which carry role tags, predicate sentiment tags and scores corresponding to at least one intent tag.

[0099] Optional, model input module 320, specifically used for:

[0100] Establish the first model;

[0101] The text sample is input into the first model to obtain the predicted role label, predicted predicate sentiment, and scores corresponding to at least one predicted intent;

[0102] The parameters of the first model are trained based on the objective function formed by the predicted role label, predicted predicate sentiment, score corresponding to at least one predicted intent, role label, predicate sentiment label and score corresponding to at least one intent label carried by the text sample;

[0103] Return to the operation of inputting the text sample into the first model to obtain the predicted role label, predicted predicate sentiment, and at least one score corresponding to the predicted intent, until the intent recognition model is obtained.

[0104] Optionally, the model input module 320 includes:

[0105] The vector acquisition unit is used to perform BERT encoding on the text to be identified to obtain the semantic information vector corresponding to each character in the text to be identified and the initial semantic information vector corresponding to the text to be identified.

[0106] The first label sequence obtaining unit is used to input the semantic information vector corresponding to each character in the text to be identified into the label determining layer to obtain the first label sequence corresponding to the text to be identified.

[0107] An attention vector generation unit is used to generate an attention vector based on the first label sequence, the semantic information vector corresponding to each character in the text to be identified, and the initial semantic information corresponding to the text to be identified.

[0108] The target fusion vector acquisition unit is used to concatenate the attention vector and the initial semantic information vector corresponding to the text to be identified to obtain the target fusion vector;

[0109] The target intent acquisition unit is used to input the target fusion vector into the activation layer to obtain the target intent.

[0110] Optional attention vector generation unit, specifically used for:

[0111] The target character extraction subunit is used to extract target characters from the text to be identified based on the first label sequence;

[0112] A semantic information vector acquisition subunit is used to acquire the semantic information vector corresponding to the target character;

[0113] The weight determination subunit is used to determine the weight of each character in the text to be identified based on the semantic information vector corresponding to the target character and the initial semantic information vector corresponding to the text to be identified.

[0114] The attention vector acquisition sub-unit is used to perform a weighted summation of the semantic information vectors corresponding to the target character based on the weights, thereby obtaining the attention vector.

[0115] Optionally, weights determine sub-units, specifically used for:

[0116] The initial semantic information vector corresponding to the text to be identified and the semantic information vector corresponding to the target character are multiplied by a dot to obtain the score for each character.

[0117] The score corresponding to each character is normalized to obtain the weight of each character.

[0118] Optionally, the first tag sequence acquisition unit is specifically used for:

[0119] The vector corresponding to each character in the text to be identified is input into the label determination layer to obtain the first label sequence corresponding to the text to be identified, including:

[0120] The vector corresponding to each character in the text to be identified is input into the CRF layer to obtain the score of at least one label sequence corresponding to the text to be identified;

[0121] The highest-scoring tag sequence among at least one tag sequence corresponding to the text to be identified is determined as the first tag sequence.

[0122] The intent recognition device provided in this embodiment of the invention can execute an intent recognition method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0123] Example 3

[0124] Figure 4A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0125] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0126] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0127] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as intent recognition methods.

[0128] In some embodiments, the intent recognition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the intent recognition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the intent recognition method by any other suitable means (e.g., by means of firmware).

[0129] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0130] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0131] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0132] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0133] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0134] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0135] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0136] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An intent recognition method, characterized in that, include: Obtain the text to be recognized; The text to be identified is input into the intent recognition model to obtain the target intent. The intent recognition model is obtained by iterating the first model through a target sample set. The target sample set includes: text samples, which carry role tags, predicate sentiment tags and scores corresponding to at least one intent tag. The text to be recognized is input into the intent recognition model to obtain the target intent, including: The text to be identified is encoded using BERT to obtain the semantic information vector corresponding to each character in the text to be identified and the initial semantic information vector corresponding to the text to be identified. The semantic information vector corresponding to each character in the text to be identified is input into the label determination layer to obtain the first label sequence corresponding to the text to be identified. An attention vector is generated based on the first label sequence, the semantic information vector corresponding to each character in the text to be identified, and the initial semantic information corresponding to the text to be identified. The attention vector and the initial semantic information vector corresponding to the text to be identified are concatenated to obtain the target fusion vector; The target fusion vector is input into the activation layer to obtain the target intent.

2. The method according to claim 1, characterized in that, Iterate the first model using the target sample set, including: Establish the first model; The text sample is input into the first model to obtain the predicted role label, predicted predicate sentiment, and scores corresponding to at least one predicted intent; The parameters of the first model are trained based on the objective function formed by the predicted role label, predicted predicate sentiment, score corresponding to at least one predicted intent, role label, predicate sentiment label and score corresponding to at least one intent label carried by the text sample; Return to the operation of inputting the text sample into the first model to obtain the predicted role label, predicted predicate sentiment, and at least one score corresponding to the predicted intent, until the intent recognition model is obtained.

3. The method according to claim 1, characterized in that, An attention vector is generated based on the first label sequence, the semantic information vector corresponding to each character in the text to be identified, and the initial semantic information corresponding to the text to be identified, including: Extract target characters from the text to be identified based on the first tag sequence; Obtain the semantic information vector corresponding to the target character; The weight of each character in the text to be identified is determined based on the semantic information vector corresponding to the target character and the initial semantic information vector corresponding to the text to be identified. The attention vector is obtained by weighted summation of the semantic information vectors corresponding to the target characters.

4. The method according to claim 3, characterized in that, The weight of each character in the text to be identified is determined based on the semantic information vector corresponding to the target character and the initial semantic information vector corresponding to the text to be identified, including: The initial semantic information vector corresponding to the text to be identified and the semantic information vector corresponding to the target character are multiplied by a dot to obtain the score for each character. The score corresponding to each character is normalized to obtain the weight of each character.

5. The method according to claim 1, characterized in that, The vector corresponding to each character in the text to be identified is input into the label determination layer to obtain the first label sequence corresponding to the text to be identified, including: The vector corresponding to each character in the text to be identified is input into the CRF layer to obtain the score of at least one label sequence corresponding to the text to be identified; The highest-scoring tag sequence among at least one tag sequence corresponding to the text to be identified is determined as the first tag sequence.

6. An intent recognition device, characterized in that, include: The acquisition module is used to acquire the text to be recognized; The model input module is used to input the text to be recognized into the intent recognition model to obtain the target intent. The intent recognition model is obtained by iterating the first model through a target sample set. The target sample set includes: text samples, which carry role tags, predicate sentiment tags and scores corresponding to at least one intent tag. The model input module includes: The vector acquisition unit is used to perform BERT encoding on the text to be identified to obtain the semantic information vector corresponding to each character in the text to be identified and the initial semantic information vector corresponding to the text to be identified. The first label sequence obtaining unit is used to input the semantic information vector corresponding to each character in the text to be identified into the label determining layer to obtain the first label sequence corresponding to the text to be identified. An attention vector generation unit is used to generate an attention vector based on the first label sequence, the semantic information vector corresponding to each character in the text to be identified, and the initial semantic information corresponding to the text to be identified. The target fusion vector acquisition unit is used to concatenate the attention vector and the initial semantic information vector corresponding to the text to be identified to obtain the target fusion vector; The target intent acquisition unit is used to input the target fusion vector into the activation layer to obtain the target intent.

7. The apparatus according to claim 6, characterized in that, The model input module is specifically used for: Establish the first model; The text sample is input into the first model to obtain the predicted role label, predicted predicate sentiment, and scores corresponding to at least one predicted intent; The parameters of the first model are trained based on the objective function formed by the predicted role label, predicted predicate sentiment, score corresponding to at least one predicted intent, role label, predicate sentiment label and score corresponding to at least one intent label carried by the text sample; Return to the operation of inputting the text sample into the first model to obtain the predicted role label, predicted predicate sentiment, and at least one score corresponding to the predicted intent, until the intent recognition model is obtained.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the intent recognition method according to any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the intent recognition method according to any one of claims 1-5.

Images