Intention recognition and model training method, electronic device and computer storage medium

Abstract

The embodiment of the present application provides an intent recognition and model training method, an electronic device and a computer storage medium, wherein the intent recognition method includes: obtaining the original feature coding sequence corresponding to the text to be recognized; masking the tags in the text to be recognized to obtain the masked text and the masked feature coding sequence corresponding to the masked text; based on the original feature coding sequence and the masked feature coding sequence, determining the structural information between each tag in the text to be recognized, and obtaining the structural feature vector corresponding to the structural information; aggregating the original feature coding sequence and the structural feature vector, and performing intent recognition of the text to be recognized based on the aggregation result. Through the scheme provided by the embodiment of the present application, more accurate intent recognition can be achieved, and the intent recognition cost is reduced, thereby improving the efficiency of intent recognition as a whole.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of artificial intelligence technology, and in particular, to an intent recognition method, an intent recognition model training method, and corresponding electronic devices and computer storage media. Background Art

[0002] Intent recognition is a basic natural language processing task that is used to identify the intent expressed in text and has a wide range of uses in the field of artificial intelligence.

[0003] At present, intent recognition mostly uses the method of sending the text into a pre-trained language model for encoding, and then calculating the intent recognition result based on the encoding result through a multi-layer perceptron (MLP). Although this method solves the problem of short sentence intent recognition to a certain extent, it is still not enough for long and difficult sentences. This is because the logical relationship between the components in long and difficult sentences is complex and has remote connections. The existing pre-trained language model cannot analyze the complex logic contained in long and difficult sentences, resulting in poor accuracy in intent recognition of long and difficult sentences.

[0004] Therefore, how to more accurately identify the intent of long and difficult sentences has become an urgent problem to be solved. Summary of the invention

[0005] In view of this, an embodiment of the present application provides an intent recognition and model training solution to at least partially solve the above-mentioned problems.

[0006] According to a first aspect of an embodiment of the present application, a method for intent recognition is provided, comprising: obtaining an original feature coding sequence corresponding to a text to be recognized; performing masking on tags in the text to be recognized to obtain masked text and a masked feature coding sequence corresponding to the masked text; determining structural information between tags in the text to be recognized based on the original feature coding sequence and the masked feature coding sequence, and obtaining a structural feature vector corresponding to the structural information; aggregating the original feature coding sequence and the structural feature vector, and performing intent recognition on the text to be recognized based on the aggregation result.

[0007] According to a second aspect of an embodiment of the present application, a method for training an intent recognition model is provided, comprising: obtaining training samples for training the intent recognition model, the training samples comprising a text sample to be recognized and an intent label corresponding to the text sample to be recognized; inputting the training samples into the intent recognition model to be trained, and obtaining an original feature coding sequence corresponding to the text sample to be recognized; performing masking on the tags in the text sample to be recognized, and obtaining a masked text sample and a masked feature coding sequence corresponding to the masked text sample; determining structural information between each tag in the text sample to be recognized based on the original feature coding sequence and the masked feature coding sequence, and obtaining a structural feature vector corresponding to the structural information; aggregating the original feature coding sequence and the structural feature vector, and performing intent prediction on the aggregation result; and training the intent recognition model according to the difference between the intent prediction result and the intent label.

[0008] According to a third aspect of an embodiment of the present application, there is provided an intention recognition method, comprising: obtaining a conversation text corresponding to a user's voice conversation data and an original feature coding sequence of the conversation text; performing masking processing on the tags in the conversation text to obtain a masked text and a masked feature coding sequence corresponding to the masked text; determining structural information between each tag in the conversation text based on the original feature coding sequence and the masked feature coding sequence, and obtaining a structural feature vector corresponding to the structural information; aggregating the original feature coding sequence and the structural feature vector, and performing conversation intention recognition according to the aggregation result.

[0009] According to a fourth aspect of an embodiment of the present application, there is provided an intent recognition method, comprising: obtaining an original feature coding sequence corresponding to a user's search request; masking the tags in the search request to obtain masked text and a masked feature coding sequence corresponding to the masked text; determining structural information between each tag in the search request based on the original feature coding sequence and the masked feature coding sequence, and obtaining a structural feature vector corresponding to the structural information; aggregating the original feature coding sequence and the structural feature vector, and performing search intent recognition for the search request based on the aggregation result.

[0010] According to the fifth aspect of an embodiment of the present application, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described above.

[0011] According to a sixth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method described above is implemented.

[0012] According to a seventh aspect of an embodiment of the present application, a computer program product is provided, comprising computer instructions, wherein the computer instructions instruct a computing device to perform operations corresponding to the method described above.

[0013] According to the solution provided in the embodiment of the present application, when performing intent recognition on a text, not only the semantic information carried by its original feature coding sequence is considered, but also the structural information of the text is obtained. Because the structural information of the text can effectively characterize the logical relationship between the various parts of the text (such as characters, words, phrases, etc.), the intent recognition performed after combining the structural information and semantic information can more accurately capture the dependencies and influence relationships between the various parts of the text, thereby achieving more accurate intent recognition. In addition, the structural information of the text is generated based on the original feature coding sequence and the masked feature coding sequence, so that the text structure information can be accurately obtained without relying on external information, which reduces the cost of intent recognition and improves the efficiency of intent recognition as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0015] Figure 1 A schematic diagram of an exemplary system for a verification code generation method applicable to an embodiment of the present application;

[0016] Figure 2A A flowchart of the steps of an intent recognition model training method according to Embodiment 1 of the present application;

[0017] Figure 2B for Figure 2A A schematic diagram of the structure of an intent recognition model in the illustrated embodiment;

[0018] Figure 3A is a flowchart of a method for identifying intention according to Embodiment 2 of the present application;

[0019] Figure 3B for Figure 3A A schematic diagram of an example scenario in the illustrated embodiment;

[0020] Figure 3C for Figure 3AA schematic diagram of another example scenario in the illustrated embodiment;

[0021] Figure 4 This is a schematic diagram of the structure of an electronic device according to the third embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0023] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0024] Figure 1 An exemplary system applicable to the embodiment of the present application is shown. Figure 1 As shown, the system 100 may include a server 102, a communication network 104 and / or one or more user devices 106. Figure 1 The example in FIG. 1 is a plurality of user devices.

[0025] The server 102 may be any suitable server for storing information, data, programs and / or any other suitable type of content. In some embodiments, the server 102 may perform any suitable function. For example, in some embodiments, the server 102 may be used to perform intent recognition on the text. As an optional example, in some embodiments, the server 102 may be used to perform text intent recognition based on the original feature coding sequence carrying semantic information corresponding to the text and the structural feature vector carrying structural information. As another example, in some embodiments, when the server 102 obtains the structural information of the text, it may obtain the structural information of the text by masking the token in the text, based on the masked feature coding sequence corresponding to the masked text and the original feature coding sequence corresponding to the original text. In some embodiments, an intent recognition model may be provided in the server 102 to realize the intent recognition of the text through the intent recognition model. Optionally, in some embodiments, the server 102 may also train the intent recognition model. In some embodiments, after the server 102 performs intent recognition on the text and obtains the intent recognition result, the intent recognition result may be sent to the user device.

[0026] In some embodiments, the communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 104 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN) and / or any other suitable communication network. The user device 106 can be connected to the communication network 104 via one or more communication links (e.g., communication link 112), and the communication network 104 can be linked to the server 102 via one or more communication links (e.g., communication link 114). The communication link can be any communication link suitable for transmitting data between the user device 106 and the server 102, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link or any suitable combination of such links.

[0027] The user device 106 may include any one or more user devices suitable for interacting with a user and receiving user input. In some embodiments, the user device 106 may include any suitable type of device. For example, in some embodiments, the user device 106 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, a wearable computer, a game console, a media player, a vehicle entertainment system, and / or any other suitable type of user device. In some embodiments, the user device 106 may send the information or data input by the user to the server 102 for the server 102 to perform intent recognition on it. In some embodiments, the user device 106 may also receive the intent recognition result returned by the server 102.

[0028] Although server 102 is illustrated as one device, in some embodiments, any suitable number of devices may be used to perform the functions performed by server 102. For example, in some embodiments, multiple devices may be used to implement the functions performed by server 102. Alternatively, the functions of server 102 may be implemented using cloud services.

[0029] Based on the above system, the present application scheme is described below through multiple embodiments. In addition, for ease of understanding, the following embodiments first describe the training of the intent recognition model, and then describe the application of the trained intent recognition model.

[0030] Embodiment 1

[0031] Reference Figure 2A , shows a step flow chart of an intent recognition model training method according to embodiment 1 of the present application.

[0032] The intention recognition model training method of this embodiment includes the following steps:

[0033] Step S202: Obtain training samples for training the intent recognition model.

[0034] The training samples include text samples to be identified and intent labels corresponding to the text samples to be identified.

[0035] The text samples to be recognized can be text samples in any field, including but not limited to: human-computer dialogue field, online customer service field, online search field, e-commerce field, online medical field, etc., and the embodiments of the present application are not limited to this.

[0036] The intent labels corresponding to the text samples to be identified can be appropriately set by those skilled in the art according to actual needs. For example, in the field of e-commerce, they can include: inquiry intent, model intent, material intent, order intent, etc.; in the field of online search, they can include more abundant intent labels, such as: address search intent, person search intent, news search intent, transaction search intent, etc. It should be noted that the above intent labels are only exemplary. In actual applications, intent labels will be richer and more complex, and intent labels can also include multiple levels. The intent labels of the next level are used to represent the subdivided intent of the intent labels of the previous level, and so on.

[0037] Step S204: input the training sample into the intention recognition model to be trained, and obtain the original feature coding sequence corresponding to the text sample to be recognized.

[0038] In order to explain the training of the intent recognition model more clearly, in this step, the structure of the intent recognition model is first explained as follows: Figure 2B as shown in .

[0039] from Figure 2BIt can be seen that the intent recognition model in the embodiment of the present application includes a first deep neural network encoder part, a second deep neural network encoder part, a graph neural network part, and a prediction output part. Among them, the first deep neural network encoder part can be implemented by an encoder with a Transformer structure (including but not limited to Transformer Encoder); the second deep neural network encoder part can be implemented by an encoder with an MLM self-supervisory task and a Transformer structure (including but not limited to Transformer Encoder); the graph neural network part can be implemented by GNN, GCN, GAT and other models; and the prediction output part can include a splicing layer and an MLP layer, the splicing layer is used to splice the original feature coding sequence and the structural feature vector corresponding to the text sample to be recognized to achieve aggregation of the two, and the MLP layer is used to output the intent prediction result based on the splicing result. In the training stage, the MLP layer will also calculate the difference between the prediction result and the intent label, and train the intent recognition model based on this difference. In the application stage, MLP will directly output the intent prediction result. For the second deep neural network encoder, MLM is a model that is trained by destroying certain mask positions in a text sequence and requiring the model to restore them. Based on MLM, masking and self-supervised learning of text sequences can be effectively implemented without relying on external label information. In this embodiment, the MLM is set to a model that has completed training.

[0040] Based on this, in this step, after the training sample is input into the intent recognition model, the text sample to be recognized is first obtained, and it is encoded by the first deep neural network encoder part to obtain the corresponding encoding sequence, which is called the original feature encoding sequence. The original feature encoding sequence can effectively represent the semantic information of the text sample to be recognized.

[0041] Step S206: performing mask processing on the marks in the text sample to be recognized to obtain the masked text sample and the masked feature coding sequence corresponding to the masked text sample.

[0042] In one feasible manner, this step can be implemented as follows: for each mark in the text sample to be identified, mask processing is performed separately to obtain multiple masked text samples with the same number as the mark; and multiple masked text samples are encoded separately to obtain corresponding multiple masked feature encoding sequences.

[0043] For example, assume that A, B, C, D, and E represent five tokens in a long and difficult sentence text (such as a text with more than a preset number of words, or a text containing more than two punctuation marks, or a text containing multiple grammatical relations and / or sentence patterns, etc.), and a token can be a character, a word, or a phrase. For each token, after masking the long and difficult sentence, five new token sequences (i.e., masked text samples) are obtained, which are:

[0044]

[0045] Specific to Figure 2B In the intent recognition model shown, the mask processing can be performed through its second deep neural network encoder to obtain the above-mentioned five masked text samples; then, the five masked text samples are encoded respectively to obtain the corresponding five new encoding vectors, that is, the masked feature encoding sequences, which are respectively represented as a', b', c', d', and e'.

[0046] Step S208: Based on the original feature coding sequence and the masked feature coding sequence, determine the structural information between the various tags in the text sample to be recognized, and obtain the structural feature vector corresponding to the structural information.

[0047] In a feasible manner, this step can be implemented as follows: for each masked token, determine the similarity between the codes corresponding to the unmasked tokens in the masked feature coding sequence corresponding to the masked text sample to which the token belongs and the codes corresponding to the original feature coding sequence; based on the similarity, determine the structural information between the masked tokens and the unmasked tokens. In this way, accurate acquisition of structural information between tokens can be effectively achieved without relying on external structural information annotation, and the cost of obtaining structural information is reduced.

[0048] When using Figure 2B When the intent recognition model is shown in , the structural information can be obtained by Figure 2BThe second deep neural network encoder shown in is implemented. After obtaining the masked feature coding sequence corresponding to a masked token, the second deep neural network encoder will calculate the similarity between the sequence and the original feature coding sequence in units of tokens (the calculation of the similarity corresponding to the masked token can be ignored), such as cosine similarity calculation, to determine the similarity between the other tokens in the two sequences except the masked token. Because if the masked token has a strong association with one or some of the other unmasked tokens, the encoding of the unmasked token will be greatly different from the original encoding due to the lack of the corresponding information of the masked token, that is, the similarity is low. Conversely, if the masked token has a weak association with one or some of the other unmasked tokens, the masked feature coding of the unmasked token will be less different from the original coding, that is, the similarity is high. Based on this, for each masked token, the syntactic dependency relationship between it and other tokens, that is, the structural information, can be determined. When the similarities of all positions in all sequences are traversed, the syntactic dependency between any two tokens in all tokens corresponding to the text samples to be recognized can be obtained.

[0049] Exemplarily, when calculating the syntactic dependency between the Nth token and other tokens, the original feature coding sequence corresponding to the original text sample to be recognized and the masked feature coding sequence corresponding to the Nth masked text sample corresponding to the Nth token are selected. For example, when calculating the syntactic dependency between the two tokens A and B, the original feature coding sequence and the masked feature coding sequence corresponding to the above sequence a are selected.

[0050] The syntactic dependency between tokens is calculated as follows: for the Nth token and the Mth token, the similarity between the feature codes at the Mth position in the two feature code sequences is calculated, such as the cosine similarity. For example, to calculate the syntactic dependency between the two tokens A and B, the cosine similarity of the position corresponding to B in the original feature code sequence and the masked feature code sequence corresponding to sequence a needs to be calculated.

[0051] After traversing all positions of all sequences, the syntactic dependency relationship between any two tokens in the original text sample to be recognized can be obtained. In a feasible manner, after obtaining the syntactic dependency relationship between the any two tokens, the edges greater than or equal to a preset similarity threshold can be retained, and the edges less than the similarity threshold can be removed, thereby completing the construction of the syntactic dependency tree associated with the text sample to be recognized.

[0052] Alternatively, for each masked token, the similarity corresponding to each unmasked token obtained based on the masked token may be screened according to a preset similarity threshold; and based on the screening result, structural information between the masked token and the token corresponding to the screened similarity may be generated.

[0053] In the above process, the similarity threshold can be appropriately set by those skilled in the art according to actual needs, such as a normalized value of 0.7, etc., and the present application embodiment does not limit this. Through the above method, those data that affect the deterministic syntactic dependency between tokens can be filtered out, and the syntactic dependency between tokens can be effectively and accurately determined, and the data processing volume and the adverse effects of noise data on syntactic dependency are greatly reduced.

[0054] Furthermore, the corresponding structural feature vector can be obtained based on the structural information. When the structural information is expressed in the form of a syntactic dependency tree, the corresponding structural feature vector can be extracted through a graph neural network model. That is, a syntactic dependency tree of the text sample to be identified can be generated based on the structural information; the syntactic dependency tree is extracted through the graph neural network model to obtain the corresponding structural feature vector. The structural feature vector is easier to process and aggregate with the original feature encoding vector in the subsequent process.

[0055] For example, when using Figure 2B When the intent recognition model shown in is used, its graph neural network (including but not limited to GNN, GCN, GAT and other specific forms of graph neural networks) can be applied to extract the structural information of the syntactic dependency tree. The extracted information contains the information of multiple graph nodes, and the information of all nodes is averaged and pooled to obtain the final structural information associated with the text sample to be recognized, that is, the structural feature vector.

[0056] Step S210: Aggregate the original feature coding sequence and the structural feature vector, and perform intent prediction on the aggregation result.

[0057] In this embodiment, the aggregation is implemented in the form of splicing, but is not limited thereto. Other forms of vector fusion are also applicable to the solution of the embodiment of the present application.

[0058] For example, the structural feature vector and the original feature coding sequence can be concatenated, and the concatenated vector carries both the semantic information of the text sample to be identified and the structural information. Then, the concatenated vector is subjected to intent prediction by a nonlinear multi-layer perceptron MLP, and is mapped to a C-dimensional vector, the dimension of which is consistent with the number of intent labels.

[0059] Step S212: training the intent recognition model based on the difference between the intent prediction result and the intent label.

[0060] After obtaining the intent prediction result, the intent prediction result and the intent label corresponding to the text sample to be recognized can be calculated based on a preset loss function to obtain the loss value, which is the difference between the two, and the intent recognition model can be trained based on the difference.

[0061] In one example, the intent label of the text sample to be identified can be implemented in the form of a one-hot code. Based on this, the cross entropy loss between the intent prediction result and the true one-hot intent label can be calculated and optimized, and the model training is performed based on the loss. Among them, the cross entropy loss is determined by the corresponding cross entropy loss function, and the cross entropy loss function can adopt a conventional cross entropy loss function. The specific implementation of the cross entropy loss function in the embodiment of the present application will not be described in detail.

[0062] The training is iterated until the training termination condition is reached, such as reaching a preset number of training times, or the loss value reaches a set threshold, etc.

[0063] It can be seen that through this embodiment, the intent recognition model obtained through training can comprehensively consider the semantic information and structural information of the text to more accurately predict the text intent. In addition, when predicting the structural information of the text, the model determines the encoding difference of the corresponding position based on the original feature encoding sequence corresponding to the token and the masked feature encoding sequence, so as to predict the structural information between tokens without relying on external label data of the structural information, which not only makes the prediction of the structural information more accurate, but also reduces the overall cost of the model implementation.

[0064] Embodiment 2

[0065] Reference Figure 3A , shows a step flow chart of an intent recognition method according to embodiment 2 of the present application.

[0066] This embodiment is based on the intention recognition model trained by the method of Embodiment 1, and focuses on explaining the intention recognition method of the embodiment of the present application from the perspective of its application.

[0067] The intention recognition method of this embodiment includes the following steps:

[0068] Step S302: Obtain the original feature coding sequence corresponding to the text to be recognized.

[0069] The text to be recognized can be text in any field that requires intent recognition, including but not limited to human-computer dialogue, online customer service, online search, e-commerce, online medical care, etc.

[0070] When implementing the solution of this embodiment based on the intention recognition model trained in the first embodiment, Figure 2B The first deep neural network encoder shown in encodes the text to be recognized to obtain the original feature encoding sequence. Its specific implementation can refer to the description of the relevant part in the first embodiment, which will not be repeated here.

[0071] Step S304: mask the marks in the text to be recognized to obtain the masked text and the masked feature coding sequence corresponding to the masked text.

[0072] In one feasible method, this step can be implemented as follows: for each marking token in the text to be recognized, mask processing is performed separately to obtain multiple masked texts with the same number as the marking token; the multiple masked texts are encoded separately to obtain corresponding multiple masked feature encoding sequences.

[0073] When implementing the solution of this embodiment based on the intention recognition model trained in the first embodiment, Figure 2B The second deep neural network encoder shown in performs mask processing on each token in the text to be recognized, and obtains multiple masked texts and multiple masked feature encoding sequences corresponding to the multiple masked texts. Its specific implementation can refer to the description of the relevant part in Example 1, which will not be repeated here.

[0074] Step S306: Based on the original feature coding sequence and the masked feature coding sequence, determine the structural information between the various tags in the text to be recognized, and obtain a structural feature vector corresponding to the structural information.

[0075] In a feasible manner, this step can be implemented as follows: for each masked token, determine the similarity between the codes corresponding to the unmasked tokens in the masked feature coding sequence corresponding to the masked text to which the token belongs and the corresponding codes in the original feature coding sequence; based on the similarity, determine the structural information between the masked token and each unmasked token.

[0076] Further optionally, based on the similarity, determining the structural information between the masked token and each unmasked token can be implemented as follows: according to a preset similarity threshold, filtering the similarity corresponding to each unmasked token; and based on the filtering result, generating the structural information between the masked token and the token corresponding to the filtered similarity.

[0077] When implementing the solution of this embodiment based on the intention recognition model trained in the first embodiment, Figure 2BThe second deep neural network encoder shown in obtains the structural information between each token in the text to be recognized. Its specific implementation can refer to the description of the relevant part in the first embodiment, which will not be repeated here.

[0078] Furthermore, a syntactic dependency tree of the text to be recognized can be generated based on the structural information between each token in the text to be recognized; then, the syntactic dependency tree is subjected to feature extraction through a graph neural network model to obtain a corresponding structural feature vector.

[0079] When implementing the solution of this embodiment based on the intention recognition model trained in the first embodiment, Figure 2B The graph neural network part shown in obtains the structural feature vector corresponding to the structural information. Its specific implementation can refer to the description of the relevant part in Example 1, which will not be repeated here.

[0080] Step S308: Aggregate the original feature coding sequence and the structural feature vector, and perform intent recognition of the text to be recognized based on the aggregation result.

[0081] Aggregation of the original feature coding sequence and the structural feature vector can be performed by splicing or fusion, thereby effectively integrating the semantic information and structural information of the text to be recognized, so as to accurately identify the intention of the text to be recognized.

[0082] When implementing the solution of this embodiment based on the intention recognition model trained in the first embodiment, Figure 2B The prediction output part shown in implements vector aggregation and intent recognition. Its specific implementation can refer to the description of the relevant parts in Example 1, which will not be repeated here. Different from the training stage in Example 1, in this embodiment, the vector value of the C-dimensional vector generated by MLP can be calculated to obtain the probability results of each type of intent in multiple preset intent categories, and the intent category with the highest probability can be taken as the final intent recognition result.

[0083] As can be seen from the above, the scheme of this embodiment first obtains the original feature coding sequence corresponding to the text to be recognized; then, the text to be recognized is masked one by one, and for any masked position, a new text sequence is generated, and it is encoded through the intention recognition model to obtain a surface representation; then, the syntactic dependency relationship between the masked position and all other positions in the sequence is analyzed, if the syntactic dependency relationship is greater than a certain threshold (i.e., a preset similarity threshold), the syntactic dependency edge is retained, otherwise the syntactic dependency edge is pruned; as the syntactic dependency relationship corresponding to each position is calculated, a syntactic dependency tree of the entire text to be recognized is also generated; further, a graph neural network is applied to the syntactic dependency tree to obtain a structural feature vector of the text to be recognized, and combined with the original feature coding sequence corresponding to the original text of the text to be recognized, the intention recognition of the text to be recognized is performed together to obtain an intention recognition result.

[0084] The above process is exemplified below by way of scenario examples.

[0085] like Figure 3B As shown, it shows a method for intention recognition in a human-computer dialogue scenario.

[0086] Specifically, the smart device receives the user's voice conversation data (which may be question data or simple non-question form of conversation exchange data, and this example does not limit the specific conversation data field or content); sends it to the server; after the server receives the voice conversation data, it converts the voice conversation data into corresponding conversation text (the smart device may also directly convert the voice conversation data into conversation text and send it to the server), and inputs it into the intention recognition model, and obtains the original feature coding sequence corresponding to the conversation text through the first neural network encoder of the intention recognition model; the marks in the conversation text are masked by the second neural network encoder of the intention recognition model to obtain the masked text and the masked feature coding sequence corresponding to the masked text; and the second neural network encoder is used to determine the structural information between each token in the conversation text based on the original feature coding sequence and the masked feature coding sequence; the structural feature vector corresponding to the structural information is obtained by the graph neural network of the intention recognition model; the original feature coding sequence and the structural feature vector are aggregated by the prediction output part of the intention recognition model, and the conversation intention is recognized according to the aggregation result, and the intention recognition result is output. Furthermore, subsequent processing can be performed based on the intention recognition result, such as determining a reply text to reply to the user's voice conversation data, and feeding the reply text back to the smart device so that the smart device can convert it into voice and play it to the user, etc.

[0087] In another scenario, Figure 3C As shown, it shows a method for intent recognition in an online search scenario.

[0088] Specifically, the smart device receives a search request input by a user and sends it to a server; after receiving the search request, the server inputs the intent recognition model, and obtains the original feature coding sequence corresponding to the search request through the first deep neural network encoder of the intent recognition model; the token in the search request is masked by the second deep neural network encoder of the intent recognition model to obtain the masked text and the masked feature coding sequence corresponding to the masked text; and the structural information between each token in the search request is determined based on the original feature coding sequence and the masked feature coding sequence by the second deep neural network encoder; the structural feature vector corresponding to the structural information is obtained by the graph neural network of the intent recognition model; the original feature coding sequence and the structural feature vector are aggregated by the prediction output part of the intent recognition model, and the search intent recognition for the search request is performed according to the aggregation result, and the intent recognition result is output. Further, subsequent processing can be performed based on the intent recognition result, such as determining the search result for the search request, and feeding the search result back to the smart device so that the smart device can display it to the user, etc.

[0089] It can be seen that through this embodiment, when performing intent recognition on a text, not only the semantic information carried by its original feature coding sequence is considered, but also the structural information of the text is obtained. Because the structural information of the text can effectively characterize the logical relationship between the various parts of the text (such as characters, words, phrases, etc.), the intent recognition performed after combining the structural information and semantic information can more accurately capture the dependencies and influence relationships between the various parts of the text, and achieve more accurate intent recognition. In addition, the structural information of the text is generated based on the original feature coding sequence and the masked feature coding sequence, so that the text structure information can be accurately obtained without relying on external information, which reduces the cost of intent recognition and improves the efficiency of intent recognition as a whole.

[0090] Embodiment 3

[0091] Reference Figure 4 , shows a schematic diagram of the structure of an electronic device according to the third embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device.

[0092] like Figure 4 As shown, the electronic device may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0093] in:

[0094] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a communication bus 408 .

[0095] The communication interface 404 is used to communicate with other electronic devices or servers.

[0096] The processor 402 is used to execute the program 410, and specifically can execute the relevant steps in any of the above method embodiments.

[0097] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0098] The processor 402 may be a CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0099] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0100] Program 410 can be specifically used to enable processor 402 to execute operations corresponding to the intent recognition method or intent recognition model training method described in any of the aforementioned method embodiments.

[0101] The specific implementation of each step in program 410 can refer to the corresponding description of the corresponding steps and units in the above method embodiment, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0102] An embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to perform operations corresponding to any intent recognition method or intent recognition model training method in the above-mentioned multiple method embodiments.

[0103] It should be noted that, in multiple embodiments of the present application, the intention recognition of long and difficult sentences is taken as an example, but those skilled in the art should understand that, in actual applications, the solutions of the embodiments of the present application are not only applicable to long and difficult sentences, but also to other sentences, such as standard sentences or short sentences.

[0104] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0105] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the method shown here.

[0106] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0107] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. An intention recognition method, comprising: Obtain the original feature encoding sequence corresponding to the text to be recognized; Masking the marks in the text to be recognized to obtain the masked text and a masked feature coding sequence corresponding to the masked text; Based on the original feature coding sequence and the masked feature coding sequence, determining the structural information between the various tags in the to-be-recognized text, and obtaining a structural feature vector corresponding to the structural information; Aggregating the original feature coding sequence and the structural feature vector, and performing intention recognition of the to-be-recognized text according to the aggregation result; Among them, masking is performed on the marks in the text to be recognized to obtain the masked text and the masked feature coding sequence corresponding to the masked text, including: using the encoder in the intention recognition model to mask the marks in the text to be recognized to obtain the masked text and the masked feature coding sequence.

2. The method according to claim 1, wherein: Using the encoder in the intention recognition model, masking the tokens in the text to be recognized to obtain the masked text and the masked feature encoding sequence, including: Using the encoder, masking is performed on each mark in the text to be recognized to obtain a plurality of masked texts having the same number as the marks; The plurality of masked texts are respectively encoded to obtain a plurality of corresponding masked feature encoding sequences.

3. The method according to claim 2, wherein: The determining, based on the original feature coding sequence and the masked feature coding sequence, structural information between each tag in the to-be-recognized text comprises: For each masked token, determine the similarity between the codes corresponding to the unmasked tokens in the masked feature code sequence corresponding to the masked text to which the token belongs and the codes corresponding to the original feature code sequence; According to the similarity, structural information between the masked markers and each marker that is not masked is determined.

4. The method according to claim 3, wherein: The determining, according to the similarity, structural information between the masked mark and each mark not masked includes: According to the preset similarity threshold, the similarity corresponding to each unmasked mark is screened; According to the screening results, structural information between the masked tags and the filtered tags corresponding to the similarity is generated.

5. The method according to any one of claims 1 to 4, wherein: The obtaining of the structural feature vector corresponding to the structural information includes: Generating a syntactic dependency tree of the text to be recognized according to the structural information; The syntactic dependency tree is subjected to feature extraction through a graph neural network model to obtain a corresponding structural feature vector.

6. A method for training an intent recognition model, comprising: Acquire a training sample for training an intent recognition model, wherein the training sample includes a text sample to be recognized and an intent label corresponding to the text sample to be recognized; Input the training sample into the intention recognition model to be trained, and obtain the original feature coding sequence corresponding to the text sample to be recognized; Masking the marks in the text sample to be recognized to obtain a masked text sample and a masked feature coding sequence corresponding to the masked text sample; Based on the original feature coding sequence and the masked feature coding sequence, determining the structural information between the various tags in the to-be-recognized text sample, and obtaining a structural feature vector corresponding to the structural information; Aggregating the original feature coding sequence and the structural feature vector, and performing intent prediction on the aggregation result; Training the intent recognition model according to the difference between the intent prediction result and the intent label; Among them, masking is performed on the marks in the text sample to be identified to obtain the masked text sample and the masked feature coding sequence corresponding to the masked text sample, including: using the encoder in the intention recognition model to mask the marks in the text sample to be identified to obtain the masked text sample and the masked feature coding sequence.

7. The method according to claim 6, wherein: Using the encoder in the intention recognition model, masking the tags in the text sample to be recognized to obtain the masked text sample and the masked feature encoding sequence, including: Using the encoder, masking is performed on each mark in the text sample to be recognized, so as to obtain a plurality of masked text samples having the same number as the marks; The plurality of masked text samples are respectively encoded to obtain a plurality of corresponding masked feature encoding sequences.

8. The method according to claim 7, wherein: The determining, based on the original feature coding sequence and the masked feature coding sequence, structural information between each tag in the to-be-recognized text sample comprises: For each masked token, determine the similarity between the codes corresponding to the unmasked tokens in the masked feature code sequence corresponding to the masked text sample to which the token belongs and the codes corresponding to the original feature code sequence; According to the similarity, structural information between the masked markers and each marker that is not masked is determined.

9. The method according to any one of claims 6 to 8, wherein: The obtaining of the structural feature vector corresponding to the structural information includes: Generating a syntactic dependency tree of the to-be-recognized text sample according to the structural information; The syntactic dependency tree is subjected to feature extraction through a graph neural network model to obtain a corresponding structural feature vector.

10. An intention recognition method, comprising: Acquire the conversation text corresponding to the user's voice conversation data and the original feature coding sequence of the conversation text; Masking the marks in the dialogue text to obtain masked text and a masked feature coding sequence corresponding to the masked text; Based on the original feature coding sequence and the masked feature coding sequence, determining the structural information between the various tags in the dialogue text, and obtaining a structural feature vector corresponding to the structural information; Aggregating the original feature coding sequence and the structural feature vector, and performing dialogue intention recognition according to the aggregation result; Among them, masking the marks in the dialogue text to obtain the masked text and the masked feature coding sequence corresponding to the masked text includes: using the encoder in the intention recognition model to mask the marks in the dialogue text to obtain the masked text and the masked feature coding sequence.

11. A method for identifying intent, comprising: Obtaining the original feature encoding sequence corresponding to the user's search request; Masking the tags in the search request to obtain masked text and a masked feature coding sequence corresponding to the masked text; Determine structural information between each tag in the search request based on the original feature coding sequence and the masked feature coding sequence, and obtain a structural feature vector corresponding to the structural information; Aggregating the original feature coding sequence and the structural feature vector, and identifying the search intent for the search request according to the aggregation result; Among them, masking is performed on the tags in the search request to obtain masked text and a masked feature coding sequence corresponding to the masked text, including: using an encoder in an intent recognition model to mask the tags in the search request to obtain the masked text and the masked feature coding sequence.

12. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1-11.

13. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

14. A computer program product, comprising computer instructions, wherein the computer instructions instruct a computing device to perform operations corresponding to the method according to any one of claims 1 to 11.

Images