Intention recognition method, device, storage medium and electronic device

Abstract

The present application discloses an intent recognition method, device, storage medium and electronic device, which relates to the field of artificial intelligence technology. The method includes: encoding a query statement to obtain query statement features, and encoding the representation information of multiple preset intent categories to obtain category representation features; performing multi-granularity matching processing based on the query statement features and the category representation features to obtain multi-granularity matching features; fusing the multi-granularity matching features to obtain fused features; analyzing according to the fused features to obtain the target intent category corresponding to the query statement. The present application can effectively identify the intent of the query, improve the query effect, and enhance the user experience.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to an intent recognition method, device, storage medium and electronic device. Background Art

[0002] Due to the diversity of user needs and document types, systems such as search systems usually need to have the ability to identify the intent of queries. For example, if the intent of a query statement is accurately identified as "consulting", the query effect can be effectively improved.

[0003] Currently, there is a way to identify the intent of a query through an intent recognition model. However, query sentences are usually very short, making it difficult to accurately obtain the intent. In addition, in many cases, a query may have different meanings. For example, "his baggage is heavy", which can mean "clothes wrapped in cloth are heavy" or "the mental burden is heavy", making it more difficult to obtain the intent. Existing intent recognition models rely heavily on training data, which makes it difficult for these intent recognition models to recognize the intent of long-tail queries that lack sufficient user supervision signals.

[0004] Therefore, there is currently a problem of poor intent recognition for queries, which results in poor query results and affects user experience. Summary of the invention

[0005] The embodiment of the present application provides an intent recognition solution that can effectively improve the intent recognition effect for queries, improve query results, and improve user experience.

[0006] The embodiments of the present application provide the following technical solutions:

[0007] According to one embodiment of the present application, an intent recognition method includes: encoding a query statement to obtain a query statement feature, and encoding representation information of multiple preset intent categories to obtain a category representation feature; performing multi-granularity matching processing based on the query statement feature and the category representation feature to obtain a multi-granularity matching feature; fusing the multi-granularity matching features to obtain a fused feature; and performing analysis based on the fused feature to obtain a target intent category corresponding to the query statement.

[0008] According to one embodiment of the present application, an intent recognition device includes: an encoding module, which is used to encode a query statement to obtain a query statement feature, and encode representation information of multiple preset intent categories to obtain a category representation feature; a matching module, which is used to perform multi-granularity matching processing based on the query statement feature and the category representation feature to obtain a multi-granularity matching feature; a fusion module, which is used to fuse the multi-granularity matching features to obtain a fused feature; and an analysis module, which is used to perform analysis based on the fused feature to obtain a target intent category corresponding to the query statement.

[0009] According to another embodiment of the present application, a storage medium stores a computer program thereon, and when the computer program is executed by a processor of a computer, the computer executes the method described in the embodiment of the present application.

[0010] According to another embodiment of the present application, an electronic device may include: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the method described in the embodiment of the present application.

[0011] According to another embodiment of the present application, a computer program product or a computer program includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0012] In an embodiment of the present application, a query statement is encoded to obtain a query statement feature, and representation information of multiple preset intent categories is encoded to obtain a category representation feature; multi-granularity matching processing is performed based on the query statement feature and the category representation feature to obtain a multi-granularity matching feature; the multi-granularity matching features are fused to obtain a fused feature; and analysis is performed based on the fused feature to obtain a target intent category corresponding to the query statement.

[0013] In this way, the representation information of the query statement and multiple preset intent categories are encoded separately to obtain the query statement features and category representation features, and further through multi-granularity matching processing, the multi-granularity matching features of the query statement and multiple preset intent categories are captured; further, the multi-granularity matching features are fused to obtain fused features that alleviate the expression gap between the query and category for intent recognition, and based on the analysis of the fused features, the target intent category of the query statement in each scenario can be accurately obtained, which effectively improves the intent recognition effect for the query, improves the query effect, and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 A flowchart of an intention recognition method according to an embodiment of the present application is shown.

[0016] Figure 2 A framework diagram of a multi-granularity matching attention network in a scenario according to the present application is shown.

[0017] Figure 3 A block diagram of an intention recognition device according to an embodiment of the present application is shown.

[0018] Figure 4 A block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are only used to explain the present disclosure and are not intended to limit the present disclosure. In addition, the embodiments provided below are partial embodiments for implementing the present disclosure, rather than providing all embodiments for implementing the present disclosure. In the absence of conflict, the technical solutions recorded in the embodiments of the present disclosure can be implemented in any combination.

[0020] It should be noted that, in the embodiments of the present disclosure, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or apparatus including a series of elements includes not only the elements explicitly recorded, but also includes other elements not explicitly listed, or also includes elements inherent to the implementation of the method or apparatus. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other related elements in the method or apparatus including the element (such as a step in the method or a unit in the apparatus, for example, a unit may be a part of a circuit, a part of a processor, a part of a program or software, etc.).

[0021] For example, the intention recognition method provided by the embodiment of the present disclosure includes a series of steps, but the intention recognition method provided by the embodiment of the present disclosure is not limited to the recorded steps. Similarly, the intention recognition device provided by the embodiment of the present disclosure includes a series of units, but the device provided by the embodiment of the present disclosure is not limited to including the units explicitly recorded, and may also include units required to obtain relevant information or perform processing based on the information.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure belongs. The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0023] Figure 1 The flowchart of the intent recognition method according to an embodiment of the present application is schematically shown. The execution subject of the intent recognition method can be a terminal or a server, the terminal is such as a TV, a computer, a mobile phone, a smart watch, and a home appliance, and the server is such as a cloud server or a physical server.

[0024] like Figure 1 As shown, the intention recognition method may include steps S110 to S140.

[0025] Step S110, encode the query statement to obtain query statement features, and encode the representation information of multiple preset intent categories to obtain category representation features; step S120, perform multi-granularity matching processing based on the query statement features and the category representation features to obtain multi-granularity matching features; step S130, fuse the multi-granularity matching features to obtain fused features; step S140, analyze according to the fused features to obtain the target intent category corresponding to the query statement.

[0026] The encoder can be used to encode the query statement and the representation information of multiple preset intent categories respectively, wherein the encoder can be a BERT model, or a CNN (Convolutional Neural Network) model, etc. In one example of the present application, the last layer of the BERT model without "CLS" can be implemented as an encoder. The encoded query statement features and category representation features can specifically be a feature vector matrix (embedding matrix).

[0027] Based on the query statement features and the category representation features, multi-granularity matching processing is performed to obtain multi-granularity matching features; the multi-granularity matching features are fused to obtain fused features. According to the fused features, the target intent category corresponding to the query statement is obtained by analyzing, specifically, the fused features are input into a preset neural network for classification to perform intent classification, and the target intent category corresponding to the query statement can be obtained.

[0028] In this way, based on steps S110 to S140, the representation information of the query statement and multiple preset intent categories are encoded respectively to obtain query statement features and category representation features, and further through multi-granularity matching processing, multi-granularity matching features of the query statement and multiple preset intent categories are captured; further, the multi-granularity matching features are fused to obtain fused features that alleviate the expression gap between the query and category for intent recognition, and based on the analysis of the fused features, the target intent category of the query statement in each scenario can be accurately obtained, which effectively improves the intent recognition effect for the query, improves the query effect, and improves the user experience.

[0029] Described below Figure 1 When performing intent recognition under the embodiment, further optional specific embodiments are provided for each step performed.

[0030] In one embodiment, before encoding the description information of the plurality of preset intent categories to obtain category representation features in step S110, the method further includes:

[0031] Obtain category names and category description information of the multiple preset intent categories; concatenate the category names and category description information of the multiple preset intent categories to obtain representation information of the multiple preset intent categories.

[0032] The category names of multiple (e.g., L, usually at least 2) preset intent categories can be expressed as n=[n 1 ,n 2 ,n 3 ,...,n L ],n 1 to n L are the category names of a preset intent category, such as “consulting category”; the category description information of multiple (such as L, usually at least 2) preset intent categories can be expressed as m=[m 1 ,m 2 ,m 3 ,...,m L ],m 1 To m L They are category description information of a preset intent category, such as "query common problems in life, mostly presented in the form of questions and answers".

[0033] The category names and category description information of multiple preset intent categories are concatenated. For example, n and m are concatenated in series to obtain the representation information of multiple preset intent categories [n 1 ,n 2 ,...,n L ,m 1 ,m 2 ,...,m L]. It can be understood that in other ways, n and m can be superimposed or information intervals can be connected in series to obtain corresponding representation information.

[0034] In some embodiments, the representation information of multiple preset intent categories only includes the category names of the multiple preset intent categories. By concatenating the category names and category description information of the multiple preset intent categories, the representation information of the multiple preset intent categories is obtained, and the representation information can more effectively describe the preset intent categories, further improving the overall intent recognition effect.

[0035] In one embodiment, step S110, encoding the query statement to obtain query statement features, and encoding the representation information of multiple preset intent categories to obtain category representation features, includes:

[0036] The query statement is encoded using a shared encoder to obtain the query statement feature; the representation information of the multiple preset intent categories is encoded using the shared encoder to obtain the category representation feature, and the query statement feature and the category representation feature are located in the same semantic space.

[0037] The shared encoder is the same encoder shared when extracting features from query statements and representation information, where the encoder can be a BERT model, or a CNN (Convolutional Neural Network) model, etc.

[0038] In one example of the present application, the last layer of the BERT model without "CLS" can be implemented as a shared encoder BERT. The query statement is encoded using the shared encoder BERT to obtain the query statement feature Q i =BERT([x 1 ,x 2 ,x 3 ,...,x l ]), where x 1 、x 2 etc. are each a word in the query sentence; a shared encoder is used to encode the representation information of multiple preset intent categories, and the category representation feature C can be obtained j =BERT([n 1 ,n 2 ,...,n L ,m 1 ,m 2 ,...,m L ]).

[0039] It can be understood that in some methods, the query statement and the representation information of multiple preset intent categories can be encoded using corresponding non-shared encoders to obtain corresponding query statement features and category representation features, respectively.

[0040] By using a shared encoder to encode the query statement and the representation information of multiple preset intent categories respectively, the corresponding query statement features and category representation features can be obtained respectively, so that the query statement features and category representation features are in the same semantic space. The relevant steps in intent recognition based on the query statement features and category representation features in the same semantic space can further improve the intent recognition effect as a whole.

[0041] Further, in one embodiment, the multi-granularity matching feature includes at least two of a self-matching feature, a character-level matching feature, and a semantic-level matching feature; step S120, performing multi-granularity matching processing based on the query statement feature and the category representation feature to obtain the multi-granularity matching feature includes at least two of the following methods:

[0042] The first method is to perform self-matching based on the query statement feature to obtain the self-matching feature, where the self-matching feature is used to characterize the importance of each word in the query statement for constituting the query statement;

[0043] The second method is to perform character-level matching based on the query statement feature and the category representation feature to obtain the character-level matching feature, where the character-level matching feature is used to represent the character-level matching feature between the query statement and each of the preset intent categories;

[0044] The third method is to perform semantic-level matching based on the query statement feature and the category representation feature to obtain the semantic-level matching feature, and the semantic-level matching feature is used to characterize the semantic-level matching feature between the query statement and each of the preset intent categories.

[0045] By performing matching processing by at least two of the first method, the second method and the third method, a multi-granularity matching feature is obtained, and the multi-granularity matching feature includes at least two of a self-matching feature, a character-level matching feature and a semantic-level matching feature.

[0046] The self-matching feature is used to characterize the importance of each word in the query statement for constituting the query statement, the character-level matching feature is used to characterize the matching features at the character level between the query statement and each of the preset intent categories, and the semantic-level matching feature is used to characterize the matching features at the semantic level between the query statement and each of the preset intent categories.

[0047] The multi-granularity matching features include at least two of the self-matching features, character-level matching features, and semantic-level matching features, which can effectively capture the multi-granularity matching features of the query statement and multiple preset intent categories. It can be understood that in an optimal example, the multi-granularity matching features include three types: self-matching features, character-level matching features, and semantic-level matching features.

[0048] Furthermore, in one implementation, the self-matching based on the query statement feature to obtain the self-matching feature may specifically include: calculating the word weight corresponding to each word in the query statement based on the query statement feature; normalizing the word weight corresponding to each word in the query statement to obtain the normalized weight corresponding to each word in the query statement; multiplying the normalized weight corresponding to each word in the query statement by the sub-weight corresponding to each word in the query statement feature to obtain the importance feature corresponding to each word, and the self-matching feature includes the importance feature corresponding to each word.

[0049] Specifically, self-matching can be performed based on the following formula to obtain the self-matching feature:

[0050]

[0051]

[0052] Among them, according to the formula Based on the query feature Q i Calculate the word weight corresponding to each word in the query statement, u i Including the word weight corresponding to each word in the query statement, v i , W q is the pre-trained network parameter; Q i It is the query statement feature.

[0053] Furthermore, through softmax(u i,t ) is a score function, through softmax(u i,t ) can normalize the word weight corresponding to each word to obtain the normalized weight corresponding to each word in the query statement, where u i,t That is u i The weight corresponding to the tth word in softmax(u i,t ) can determine the importance of each word in forming the sentence representation about the current query sentence.

[0054] Furthermore, the normalized weight corresponding to each word in the query statement is multiplied by the sub-weight corresponding to each word in the query statement feature. For example, the normalized weight softmax(u i,t ) and the tth word in the query sentence feature Qi The corresponding sub-weight Q i,t Multiply them together to get the important feature Q i,t softmax(u i,t ). Finally, we get q i That is, the self-matching feature, q i Includes the importance features of each word. q That is, it can be the total number of words in the query statement.

[0055] Furthermore, in one implementation, the character-level matching is performed based on the query statement feature and the category representation feature to obtain the character-level matching feature, which may specifically include: performing a dot product between the query statement feature and the category representation feature to obtain an interaction matrix between the query statement and each preset intent category; superimposing the interaction matrix between the query statement and each preset intent category on the channel dimension to obtain a character-level interaction feature map between the query statement and each preset intent category; performing convolution and pooling on the character-level interaction feature map to obtain a pooled feature map; and obtaining the character-level matching feature according to the pooled feature map.

[0056] Specifically, in order to extract fine-grained interaction features between query statements and multiple preset intent categories, the query statement feature Q i and category representation feature C j Perform dot product between them to obtain the interaction matrix M between the query statement and each preset intent category j , j is the jth preset intent category. Then the interaction matrix between the query statement and each preset intent category is superimposed on the channel dimension to obtain the character-level interaction feature map M. Specifically, it can be formulated as:

[0057]

[0058] M=[M 1 ,M 2 ,...,M C ]

[0059] Among them, W qc ∈R d*d are the pre-trained network parameters, is the interaction matrix between the query statement and the jth preset intent category, is the character-level interaction feature map between the query and each preset intent category, L q That is, it can be the total number of words in the query statement, L c Can be C j The total number of neutron features or the total number of multiple preset intent categories, C can be the total number of interaction matrices.

[0060] Then, to extract fine-grained matching features from the character-level interaction feature map M, a window size of r can be used. w The 2D convolution module of *r performs convolution on the character-level interaction feature map M, as shown in the following formula:

[0061]

[0062] Among them, k represents the number of convolution kernels, W a,b is the convolution kernel, b is the bias vector, and the formula is used to perform convolution operation on each channel of the character-level interaction feature map M to obtain the convolution feature map s.

[0063] Next, a 2D max pooling layer is applied to extract important features from the convolutional feature map s, which can be formulated as follows:

[0064]

[0065] Among them, p w and p h is the width and height of the 2D maximum pooling, and the pooling feature map is obtained by pooling through this formula.

[0066] Finally, the character-level matching features can be obtained based on the pooled feature map. In one approach, the pooled features can be directly used as the character-level matching features Z 1 Optionally, in a specific embodiment of the present application, the pooled feature map is flattened and mapped to a low-dimensional space through a linear transformation layer, and the character-level matching feature obtained can be expressed as Z 1 ,Z 1 A semantic-level fine-grained matching feature between the query sentence and each of the preset intent categories.

[0067] Furthermore, in one implementation, the semantic-level matching is performed based on the query statement features and the category representation features to obtain the semantic-level matching features, which may specifically include: performing average pooling on the category representation features to obtain category pooling features corresponding to each preset intent category; stacking the category pooling features corresponding to each preset intent category to obtain stacked features; and performing cross-attention operations on the stacked features and the query statement features to obtain the semantic-level matching features.

[0068] Specifically, for example, "apple watch" and "apple purse" both have a high correlation with "apple" at the character level, but are semantically different from each other. Therefore, capturing the correlation between the query sentence and the preset intent category at the semantic level can further improve the intent recognition effect.

[0069] To this end, in the category representation feature C iAverage pooling is used on the time step of the character sequence to average pool the category representation features and obtain the category pooling feature c corresponding to the jth preset intent category. j , j belongs to 1 to C; and the category pooling feature c corresponding to each preset intent category 1 ,c 2 ,...,c C Stacked together, we get the stacking feature D = [c 1 ,c 2 ,...,c C ]. can be formulated as follows:

[0070] c j =mean(C j ), D = [c 1 ,c 2 ,...,c C ].

[0071] Where D∈R C*d It is the representation of pooled features of all categories.

[0072] Then, applying the cross attention layer can combine the stacked features D with the query features Q i Perform cross attention operation to obtain the integrated semantic level matching feature Z 2 , which can be formulated as follows:

[0073]

[0074] Among them, W qs are the pre-trained network parameters, Z 2 It is a semantic level matching feature.

[0075] In one embodiment, the multi-granularity matching features include self-matching features, character-level matching features and semantic-level matching features; step S130, fusing the multi-granularity matching features to obtain a fused feature, including: multiplying the self-matching feature, the character-level matching feature and the semantic-level matching feature to obtain a product feature; and obtaining the fused feature based on the product feature.

[0076] In this embodiment, the multi-granularity matching features include three types: self-matching features, character-level matching features, and semantic-level matching features. The self-matching features, character-level matching features, and semantic-level matching features can be multiplied by matrix multiplication to obtain product features, and fusion features are obtained based on the product features. The fusion features can effectively alleviate the expression gap between the query and the category of intent recognition.

[0077] Specifically, in one implementation of the present application, a nonlinear transformation layer is introduced, which is defined as: Through this nonlinear transformation layer, the self-matching feature q can be transformed by matrix multiplication. i , character-level matching feature Z 1 and semantic level matching feature Z 2 Fusion, get the product feature e, where W qf , W X and W z is the pre-trained network parameter. Furthermore, the product feature can be directly used as the fusion feature or the product feature can be used as the fusion feature after being processed.

[0078] It can be understood that in other embodiments, the multi-granularity matching features include any two of self-matching features, character-level matching features and semantic-level matching features; step S130, the multi-granularity matching features are fused to obtain a fused feature, including: multiplying any two of the self-matching features, character-level matching features and semantic-level matching features to obtain a product feature; and obtaining the fused feature based on the product feature.

[0079] The above embodiment is further described below in conjunction with a scheme for performing intent recognition in a scenario. In this scenario, intent recognition is performed by applying the above embodiment of the present application. In this scenario, the above embodiment of the present application is applied in the form of a "multi-granularity matching attention network" to perform intent classification. Figure 2 , the following is a detailed description of the "multi-granularity matching attention network".

[0080] See also Figure 2 In this scenario, the “multi-granularity matching attention network” includes a “shared encoder BERT” 210, a self-matching unit 220, a character-level matching unit 230, a semantic-level matching unit 240, a fusion unit 250, and a classification unit 260.

[0081] In the "shared encoder BERT" 210, the following can be done: use the shared encoder BERT to encode the query statement query to obtain query statement features; use the shared encoder BERT to encode the representation information of multiple preset intent categories (category 1 to category N) to obtain category representation features, and the query statement features and category representation features are in the same semantic space.

[0082] In this scenario, the last layer of the BERT model without "CLS" is implemented as a shared encoder BERT. The shared encoder BERT is used to encode the query sentence query, and the query sentence feature Q can be obtained. i =BERT([x 1 ,x 2 ,x 3 ,...,x l ]), where x 1 、x2 etc. are each a word in the query sentence; a shared encoder is used to encode the representation information of multiple preset intent categories, and the category representation feature C can be obtained j =BERT([n 1 ,n 2 ,...,n L ,m 1 ,m 2 ,...,m L ]).

[0083] The self-matching unit 220 may: perform self-matching based on the query statement feature to obtain a self-matching feature, where the self-matching feature is used to characterize the importance of each word in the query statement to the query statement. The self-matching based on the query statement feature to obtain the self-matching feature may specifically include: calculating the word weight corresponding to each word in the query statement based on the query statement feature; normalizing the word weight corresponding to each word in the query statement to obtain the normalized weight corresponding to each word in the query statement; multiplying the normalized weight corresponding to each word in the query statement by the sub-weight corresponding to each word in the query statement feature to obtain the importance feature corresponding to each word, where the self-matching feature includes the importance feature corresponding to each word.

[0084] Specifically, self-matching can be performed based on the following formula to obtain the self-matching feature:

[0085]

[0086]

[0087] Among them, according to the formula Based on the query feature Q i Calculate the word weight corresponding to each word in the query statement, u i Including the word weight corresponding to each word in the query statement, v i , W q is the pre-trained network parameter; Q i It is the query statement feature.

[0088] Furthermore, through softmax(u i,t ) is a score function, through softmax(u i,t ) can normalize the word weight corresponding to each word to obtain the normalized weight corresponding to each word in the query statement, where u i,t That is u i The weight corresponding to the tth word in softmax(u i,t ) can determine the importance of each word in forming the sentence representation about the current query sentence.

[0089] Furthermore, the normalized weight corresponding to each word in the query statement is multiplied by the sub-weight corresponding to each word in the query statement feature. For example, the normalized weight softmax(u i,t ) and the tth word in the query sentence feature Q i The corresponding sub-weight Q i,t Multiply them together to get the important feature Q i,t softmax(u i,t ). Finally, we get q i That is, the self-matching feature, q i Includes the importance features of each word. q That is, it can be the total number of words in the query statement.

[0090] The character-level matching unit 230 may: perform character-level matching based on the query statement feature and the category representation feature to obtain the character-level matching feature, wherein the character-level matching feature is used to represent the character-level matching feature between the query statement and each of the preset intent categories.

[0091] The character-level matching based on the query statement feature and the category representation feature is performed to obtain the character-level matching feature, which may specifically include: performing a dot product between the query statement feature and the category representation feature to obtain an interaction matrix between the query statement and each preset intent category; superimposing the interaction matrix between the query statement and each preset intent category on the channel dimension to obtain a character-level interaction feature map between the query statement and each preset intent category; performing convolution and pooling on the character-level interaction feature map to obtain a pooled feature map; and obtaining the character-level matching feature according to the pooled feature map.

[0092] Specifically, in order to extract fine-grained interaction features between query statements and multiple preset intent categories, the query statement feature Q i and category representation feature C j Perform dot product between them to obtain the interaction matrix M between the query statement and each preset intent category j , j is the jth preset intent category. Then the interaction matrix between the query statement and each preset intent category is superimposed on the channel dimension to obtain the character-level interaction feature map M. Specifically, it can be formulated as:

[0093]

[0094] M=[M 1 ,M 2 ,...,M C ]

[0095] Among them, Wqc ∈R d*d are the pre-trained network parameters, is the interaction matrix between the query statement and the jth preset intent category, is the character-level interaction feature map between the query and each preset intent category, L q That is, it can be the total number of words in the query statement, L c Can be C j The total number of neutron features or the total number of multiple preset intent categories, C can be the total number of interaction matrices.

[0096] Then, to extract fine-grained matching features from the character-level interaction feature map M, a window size of r can be used. w The 2D convolution module of *r performs convolution on the character-level interaction feature map M, as shown in the following formula:

[0097]

[0098] Among them, k represents the number of convolution kernels, W a,b is the convolution kernel, b is the bias vector, and the formula is used to perform convolution operation on each channel of the character-level interaction feature map M to obtain the convolution feature map s.

[0099] Next, a 2D max pooling layer is applied to extract important features from the convolutional feature map s, which can be formulated as follows:

[0100]

[0101] Among them, p w and p h is the width and height of the 2D maximum pooling, and the pooling feature map is obtained by pooling through this formula.

[0102] Finally, the character-level matching features can be obtained based on the pooled feature map. In one approach, the pooled features can be directly used as the character-level matching features Z 1 Optionally, in a specific embodiment of the present application, the pooled feature map is flattened and mapped to a low-dimensional space through a linear transformation layer, and the character-level matching feature obtained can be expressed as Z 1 ,Z 1 A semantic-level fine-grained matching feature between the query sentence and each of the preset intent categories.

[0103] The semantic level matching unit 240 can: perform semantic level matching based on the query statement feature and the category representation feature to obtain the semantic level matching feature, and the semantic level matching feature is used to characterize the semantic level matching feature between the query statement and each of the preset intent categories.

[0104] The method of performing semantic-level matching based on the query statement feature and the category representation feature to obtain the semantic-level matching feature may specifically include: performing average pooling on the category representation feature to obtain category pooling features corresponding to each preset intent category; stacking the category pooling features corresponding to each preset intent category to obtain stacked features; and performing cross-attention operation on the stacked features and the query statement feature to obtain the semantic-level matching feature.

[0105] Specifically, for example, "apple watch" and "apple purse" both have a high correlation with "apple" at the character level, but are semantically different from each other. Therefore, capturing the correlation between the query sentence and the preset intent category at the semantic level can further improve the intent recognition effect.

[0106] To this end, in the category representation feature C i Average pooling is used on the time step of the character sequence to average pool the category representation features and obtain the category pooling feature c corresponding to the jth preset intent category. j , j belongs to 1 to C; and the category pooling feature c corresponding to each preset intent category 1 ,c 2 ,...,c C Stacked together, we get the stacking feature D = [c 1 ,c 2 ,...,c C ]. can be formulated as follows:

[0107] c j =mean(C j ), D = [c 1 ,c 2 ,...,c C ].

[0108] Where D∈R C*d It is the representation of pooled features of all categories.

[0109] Then, applying the cross attention layer can combine the stacked features D with the query features Q i Perform cross attention operation to obtain the integrated semantic level matching feature Z 2 , which can be formulated as follows:

[0110]

[0111] Among them, W qs are the pre-trained network parameters, Z 2 It is a semantic level matching feature.

[0112] The fusion unit 250 may: fuse the multi-granularity matching features to obtain a fused feature, including: multiplying the self-matching feature, the character-level matching feature and the semantic-level matching feature to obtain a product feature; and obtain the fused feature according to the product feature.

[0113] In this scenario, multi-granularity matching features include self-matching features, character-level matching features, and semantic-level matching features. Self-matching features, character-level matching features, and semantic-level matching features can be multiplied by matrix multiplication to obtain product features, and fusion features can be obtained based on the product features. Fusion features can effectively alleviate the expression gap between queries and categories for intent recognition.

[0114] Specifically, under this manufacturer, a nonlinear transformation layer is introduced, which is defined as: Through this nonlinear transformation layer, the self-matching feature q can be transformed by matrix multiplication. i , character-level matching feature Z 1 and semantic level matching feature Z 2 Fusion, get the product feature e, where W qf , W x and W z is the pre-trained network parameter. Furthermore, the product feature can be directly used as the fusion feature.

[0115] The classification unit 260 can: perform analysis based on the fusion features to obtain the target intent category corresponding to the query statement. Specifically, the fusion features are input into a preset neural network for classification to perform intent classification, and the target intent category corresponding to the query statement can be obtained. The target intent category is one of a plurality of preset intent categories.

[0116] Among them, in this scenario, for the entire framework of the "multi-granularity matching attention network", a variety of preset intent categories can be used for training losses. The network parameters in the "multi-granularity matching attention network" are obtained by training the entire framework in advance. The network parameters, such as the parameters in the shared encoder, v i , W q , W qc , W qs , W qf , W X and W z etc.

[0117] In this scenario, the entire framework uses multiple preset intent categories as labels in advance, and is trained by training cross entropy loss (MCE Loss), which can be formulated as follows: Among them, Y represents the predicted label (i.e., the predicted intent category) predicted by the “multi-granularity matching attention network” for the training sample, and y represents the true label corresponding to the training sample (i.e., the true intent category).

[0118] In this scenario, based on the "multi-granularity matching attention network", the representation information of the query statement and multiple preset intent categories are encoded respectively to obtain the query statement features and category representation features, and further through multi-granularity matching processing, the multi-granularity matching features of the query statement and multiple preset intent categories are captured; further, the multi-granularity matching features are fused to obtain fused features that alleviate the expression gap between the query and category for intent recognition, and based on the analysis of the fused features, the target intent category of the query statement in each scenario can be accurately obtained, which effectively improves the intent recognition effect for the query, improves the query effect, and improves the user experience.

[0119] In order to facilitate better implementation of the intention recognition method provided in the embodiment of the present application, the embodiment of the present application also provides an intention recognition device based on the above intention recognition method. The meanings of the nouns are the same as those in the above intention recognition method, and the specific implementation details can refer to the description in the method embodiment. Figure 3 A block diagram of an intention recognition device according to an embodiment of the present application is shown.

[0120] like Figure 3 As shown, the intent recognition device 300 may include: an encoding module 310 can be used to encode a query statement to obtain a query statement feature, and encode the representation information of multiple preset intent categories to obtain a category representation feature; a matching module 320 can be used to perform multi-granularity matching processing based on the query statement feature and the category representation feature to obtain a multi-granularity matching feature; a fusion module 330 can be used to fuse the multi-granularity matching features to obtain a fused feature; an analysis module 340 can be used to perform analysis based on the fused feature to obtain a target intent category corresponding to the query statement.

[0121] In some embodiments, the device also includes a splicing module, which is used to: obtain category names and category description information of the multiple preset intent categories; splice the category names and category description information of the multiple preset intent categories to obtain representation information of the multiple preset intent categories.

[0122] In some embodiments, the encoding module 310 can be used to: use a shared encoder to encode the query statement to obtain the query statement feature; use the shared encoder to encode the representation information of the multiple preset intent categories to obtain the category representation feature, and the query statement feature and the category representation feature are located in the same semantic space.

[0123] In some embodiments, the multi-granularity matching features include at least two of self-matching features, character-level matching features, and semantic-level matching features; the matching module 320 can be used to implement at least two of the following methods: performing self-matching based on the query statement features to obtain the self-matching features, and the self-matching features are used to characterize the importance of each word in the query statement for constituting the query statement; performing character-level matching based on the query statement features and the category representation features to obtain the character-level matching features, and the character-level matching features are used to characterize the character-level matching features between the query statement and each of the preset intent categories; performing semantic-level matching based on the query statement features and the category representation features to obtain the semantic-level matching features, and the semantic-level matching features are used to characterize the semantic-level matching features between the query statement and each of the preset intent categories.

[0124] In some embodiments, the matching module 320 can be used to: perform a dot product of the query statement feature and the category representation feature to obtain an interaction matrix between the query statement and each preset intent category; superimpose the interaction matrix between the query statement and each preset intent category on the channel dimension to obtain a character-level interaction feature map between the query statement and each preset intent category; perform convolution and pooling on the character-level interaction feature map to obtain a pooled feature map; and obtain the character-level matching feature based on the pooled feature map.

[0125] In some embodiments, the matching module 320 can be used to: perform average pooling on the category representation features to obtain category pooling features corresponding to each preset intent category; stack the category pooling features corresponding to each preset intent category to obtain stacked features; perform cross-attention operation on the stacked features and the query statement features to obtain the semantic-level matching features.

[0126] In some embodiments, the multi-granularity matching features include self-matching features, character-level matching features, and semantic-level matching features; the fusion module 330 can be used to: multiply the self-matching features, the character-level matching features, and the semantic-level matching features to obtain product features; and obtain the fusion features based on the product features.

[0127] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0128] In addition, the present application also provides an electronic device, such as Figure 4 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0129] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will appreciate that Figure 4 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0130] in:

[0131] The processor 401 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, it executes various functions of the computer device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0132] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0133] The electronic device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0134] The electronic device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0135] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 402 according to the following instructions, and the processor 401 will run the computer programs stored in the memory 402, thereby realizing various functions in the aforementioned embodiments of the present application.

[0136] For example, processor 401 may execute the following steps: encode a query statement to obtain a query statement feature, and encode representation information of multiple preset intent categories to obtain a category representation feature; perform multi-granularity matching processing based on the query statement feature and the category representation feature to obtain a multi-granularity matching feature; fuse the multi-granularity matching features to obtain a fused feature; and perform analysis based on the fused feature to obtain a target intent category corresponding to the query statement.

[0137] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0138] To this end, an embodiment of the present application further provides a storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any method provided in the embodiment of the present application.

[0139] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0140] Since the computer program stored in the storage medium can execute the steps in any method provided in the embodiments of the present application, the beneficial effects that can be achieved by the method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0141] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0142] It should be understood that the present application is not limited to the embodiments that have been described above and shown in the accompanying drawings, but various modifications and changes may be made without departing from the scope thereof.

Claims

1. A method for intention recognition, characterized in that: include: Encode the query sentence to obtain query sentence features, and encode the representation information of multiple preset intent categories to obtain category representation features; Calculating a word weight corresponding to each word in the query statement based on the query statement feature, and normalizing the word weight corresponding to each word in the query statement to obtain a normalized weight corresponding to each word in the query statement; Multiplying the normalized weight corresponding to each word in the query statement by the sub-weight corresponding to each word in the query statement feature to obtain the importance feature corresponding to each word, and obtaining the self-matching feature of the query statement according to the importance feature corresponding to each word; Performing dot product processing on the query statement feature and multiple category representation features to obtain an interaction matrix between the query statement and each preset intent category, and obtaining character-level matching features between the query statement and the multiple preset intent categories based on the interaction matrix, wherein the character-level matching features include matching features between the query statement and each preset intent category; Performing cross-attention calculation based on the query statement feature and the category representation feature to obtain a semantic level matching feature, wherein the semantic level matching feature is used to characterize the semantic level correlation between the query statement and each of the preset intent categories; Fusing at least two of the self-matching feature, the character-level matching feature, and the semantic-level matching feature to obtain a fused feature; An analysis is performed based on the fusion features to obtain a target intent category corresponding to the query statement.

2. The method according to claim 1, characterized in that Before encoding the description information of the plurality of preset intent categories to obtain the category representation features, the method further includes: Obtaining category names and category description information of the multiple preset intent categories; The category names and category description information of the multiple preset intent categories are concatenated to obtain representation information of the multiple preset intent categories.

3. The method according to claim 1, characterized in that The query statement is encoded to obtain query statement features, and the representation information of multiple preset intent categories is encoded to obtain category representation features, including: Encoding the query statement using a shared encoder to obtain the query statement feature; The shared encoder is used to encode the representation information of the multiple preset intent categories to obtain the category representation features, and the query statement features and the category representation features are located in the same semantic space.

4. The method according to claim 1, characterized in that: The performing character-level matching based on the query statement feature and the category representation feature to obtain the character-level matching feature includes: Performing a dot product of the query statement feature and the category representation feature to obtain an interaction matrix between the query statement and each preset intent category; Superimposing the interaction matrix between the query statement and each preset intent category on the channel dimension to obtain a character-level interaction feature map between the query statement and each preset intent category; Convolving and pooling the character-level interaction feature map to obtain a pooled feature map; The character-level matching feature is obtained according to the pooled feature map.

5. The method according to claim 1, characterized in that The performing semantic level matching based on the query statement feature and the category representation feature to obtain the semantic level matching feature includes: Performing average pooling on the category representation features to obtain category pooling features corresponding to each preset intent category; The category pooling features corresponding to each preset intent category are stacked to obtain stacked features; A cross-attention operation is performed on the stacked feature and the query sentence feature to obtain the semantic-level matching feature.

6. The method according to claim 1, characterized in that Multi-granularity matching features include self-matching features, character-level matching features, and semantic-level matching features; The multi-granularity matching features are fused to obtain fused features, including: Multiplying the self-matching feature, the character-level matching feature and the semantic-level matching feature to obtain a product feature; The fusion feature is obtained according to the product feature.

7. An intention recognition device, characterized in that: include: An encoding module, used to encode the query statement to obtain query statement features, and to encode the representation information of multiple preset intent categories to obtain category representation features; A matching module, configured to calculate a word weight corresponding to each word in the query statement based on the query statement feature, and normalize the word weight corresponding to each word in the query statement to obtain a normalized weight corresponding to each word in the query statement; Multiplying the normalized weight corresponding to each word in the query statement by the sub-weight corresponding to each word in the query statement feature to obtain the importance feature corresponding to each word, and obtaining the self-matching feature of the query statement according to the importance feature corresponding to each word; Performing dot product processing on the query statement feature and multiple category representation features to obtain an interaction matrix between the query statement and each preset intent category, and obtaining character-level matching features between the query statement and the multiple preset intent categories based on the interaction matrix, wherein the character-level matching features include matching features between the query statement and each preset intent category; Performing cross-attention calculation based on the query statement feature and the category representation feature to obtain a semantic level matching feature, wherein the semantic level matching feature is used to characterize the semantic level correlation between the query statement and each of the preset intent categories; A fusion module, used for fusing at least two of the self-matching feature, the character-level matching feature and the semantic-level matching feature to obtain a fusion feature; The analysis module is used to analyze according to the fusion features to obtain the target intent category corresponding to the query statement.

8. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: include: a memory storing a computer program; A processor reads a computer program stored in a memory to execute the method according to any one of claims 1 to 6.

Images