Intention recognition method and device, electronic equipment and storage medium
Through a single model, the input corpus is encoded and hierarchical semantic vector extraction is solved, combining the prediction model and conditional random field, the problem of low complexity and accuracy of intention recognition in the existing technology is solved, and efficient multi-level intention recognition and accurate semantics are achieved. understand.
Patent Information
- Application Number
- CN202410986844.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-29
AI Technical Summary
In the prior art, the implementation of intention recognition is complex and has low accuracy, especially when facing undefined corpus, it is prone to misidentification, and the independence between multiple models increases the system complexity and operation process.
A single model is used to encode the input corpus, extract hierarchical semantic vectors using parallel hidden layers, and combine prediction models and conditional random fields to determine the intention of the input corpus, reducing system complexity and improving recognition efficiency.
The recognition of multi-level semantic intents is realized, the error recognition rate is reduced, the recognition accuracy is improved, the intention recognition process is simplified, the user experience is improved, and the application value of semantic understanding is expanded.
Smart Images

Figure CN119047475A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to an intent recognition method, device, electronic device and storage medium. Background Art
[0002] Natural language processing is an important direction in the field of computer science. It studies how to make computers understand and generate human language. Among them, semantic understanding is the key link in natural language processing. Its purpose is to make computers understand the meaning of language, which is an important part of human-computer voice interaction.
[0003] Semantic intent often has multiple levels of parsing dimensions. In related technologies, multiple intent recognition models are usually used to identify semantic intents at different levels. Although this method can avoid confusion between multiple intents, it increases the complexity of the system and the operation process. At the same time, since multiple models are independent of each other and the model can always give a prediction result, it is easy to make misrecognitions when facing undefined corpus. Summary of the invention
[0004] In view of this, the embodiments of the present application provide an intent recognition method, device, electronic device and storage medium to solve the problem that intent recognition in the prior art is complex and has low accuracy.
[0005] A first aspect of an embodiment of the present application provides an intent recognition method, comprising:
[0006] Obtaining input corpus, encoding the input corpus, and obtaining encoded input corpus;
[0007] Use N trained parallel hidden layers to extract features from the encoded input corpus to obtain N hierarchical semantic vectors, where N is a positive integer, and each hierarchical semantic vector represents the semantic information of the input corpus at a semantic level;
[0008] Using the trained prediction model to predict the N hierarchical semantic vectors, a target prediction result is obtained, where the target prediction result includes target hierarchical semantic features in at least two different hierarchical semantic vectors, and a connection relationship between the target hierarchical semantic features;
[0009] The intent of the input corpus is determined using the hierarchical semantic features of each target and their connections.
[0010] According to a second aspect of an embodiment of the present application, there is provided an intention recognition device, comprising:
[0011] an encoding module, configured to obtain input corpus, encode the input corpus, and obtain encoded input corpus;
[0012] A feature extraction module is configured to use N trained parallel hidden layers to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors, where N is a positive integer, and each hierarchical semantic vector represents semantic information of the input corpus at a semantic level;
[0013] A prediction module is configured to use the trained prediction model to predict the N hierarchical semantic vectors to obtain a target prediction result, wherein the target prediction result includes target hierarchical semantic features in at least two different hierarchical semantic vectors and a connection relationship between the target hierarchical semantic features;
[0014] The determination module is configured to determine the intention of the input corpus using each target hierarchical semantic feature and its connection relationship.
[0015] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
[0016] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0017] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application first encode the input corpus, then use N trained parallel hidden layers to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors, then use the trained prediction model to predict the N hierarchical semantic vectors to obtain prediction results, and finally determine the intent of the input corpus based on the hierarchical semantic features of each target in the prediction results and their connection relationships, and use a single model to realize the recognition of multi-level semantic intents, thereby reducing system complexity, shortening the intent recognition process, improving recognition efficiency, and enhancing user experience.
[0018] At the same time, the intent finally identified in the technical solution provided in the embodiment of the present application is composed of multiple levels of semantic intent, and hierarchical intent recognition can be achieved based on the relationship between each level, and undefined corpus can be rejected, thereby reducing inter-class confusion in multi-classification problems, reducing the misrecognition rate, and improving recognition accuracy.
[0019] Furthermore, the technical solution provided in the embodiment of the present application combines the pre-classification model (Bidirectional Encoder Representations from Transformers, BERT) and conditional random fields (conditional random fields, CRF), which can more accurately understand the meaning of language and improve the accuracy of semantic understanding by relying on the correlation between multi-level intentions, so that the solution can not only be used for semantic understanding in human-computer interaction, but also can be widely used in information extraction, intelligent recommendation and other fields, and has broad application value. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0021] Figure 1 It is a flowchart of an intent recognition method provided in an embodiment of the present application.
[0022] Figure 2 It is a flowchart of a method for encoding input corpus to obtain encoded input corpus provided in an embodiment of the present application.
[0023] Figure 3 It is a flowchart of a method provided in an embodiment of the present application for extracting features from encoded input corpus using N trained parallel hidden layers to obtain N hierarchical semantic vectors.
[0024] Figure 4 It is a flowchart of a method provided in an embodiment of the present application for using a trained prediction model to predict N hierarchical semantic vectors to obtain a target prediction result.
[0025] Figure 5 It is a flowchart of a method provided in an embodiment of the present application for using a trained prediction model to predict the intention distribution probability and obtain a target prediction result.
[0026] Figure 6 It is a schematic diagram of using CRF to predict the probability of intent distribution provided in an embodiment of the present application.
[0027] Figure 7 It is a flowchart of a method for determining the intent of an input corpus using the hierarchical semantic features of each target and their connection relationships provided in an embodiment of the present application.
[0028] Figure 8It is a structural diagram of the global model of intent recognition provided in an embodiment of the present application.
[0029] Fig. 9 It is a flowchart of the training method of the global model of intent recognition provided in an embodiment of the present application.
[0030] Fig.10 It is a flowchart of another intent recognition method provided in an embodiment of the present application.
[0031] Fig.11 It is a schematic diagram of an intention recognition device provided in an embodiment of the present application.
[0032] Fig.12 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0033] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0034] A method and device for identifying intent according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.
[0035] As mentioned above, semantic intents often have multiple levels of parsing dimensions. For example, "turn on the air conditioner" can be seen as a "control domain", specifically "control the device" and "the device is an air conditioner". Generally, simple classification models will often cause confusion between different intents when the dimensions are not unified enough.
[0036] In related technologies, multiple intent recognition models are usually used to identify semantic intents at different levels. That is, multiple models are used, and one model classifies and obtains semantic information of one dimension, and the models in the overall system form a tree structure. For example, in the above example, a domain model is first used to obtain the domain to which it belongs as "control domain", and then the understanding model of the control domain obtains its intent as "operate the air conditioner and perform the opening operation".
[0037] Although this method can avoid confusion between multiple intents, it increases the complexity of the system and the operation process. At the same time, because multiple models are independent of each other and the model can always give a prediction result, it is easy to misidentify when facing undefined corpus.
[0038] In view of this, an embodiment of the present application provides an intent recognition method, which first encodes the input corpus, then uses N trained parallel hidden layers to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors, and then uses the trained prediction model to predict the N hierarchical semantic vectors to obtain prediction results. Finally, the intent of the input corpus is determined based on the hierarchical semantic features of each target in the prediction results and their connection relationships. A single model is used to realize the recognition of multi-level semantic intents, which reduces the system complexity, shortens the intent recognition process, improves the recognition efficiency, and enhances the user experience.
[0039] Figure 1 is a flow chart of an intention recognition method provided in an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:
[0040] In step S101, an input corpus is obtained, and the input corpus is encoded using a trained pre-trained model to obtain an encoded input corpus.
[0041] In step S102, N trained parallel hidden layers are used to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors.
[0042] Where N is a positive integer, and each hierarchical semantic vector represents the semantic information of the input corpus at a semantic level.
[0043] In step S103, the trained prediction model is used to predict the N hierarchical semantic vectors to obtain a target prediction result.
[0044] The target prediction result includes target hierarchical semantic features in at least two different hierarchical semantic vectors, and a connection relationship between the target hierarchical semantic features.
[0045] In step S104, the intention of the input corpus is determined using each target hierarchical semantic feature and its connection relationship.
[0046] In an embodiment of the present application, the intention recognition method can be executed by a server, or by a terminal device with certain computing capabilities.
[0047] In an embodiment of the present application, the server or terminal device may first obtain input corpus. In one example, the input corpus may be, for example, a voice command sent by a user, or a text corpus, or user portrait information, etc., which is not limited here. Furthermore, the input voice command, text corpus, user portrait information and other corpus information may be natural language corpus information.
[0048] In an embodiment of the present application, a trained pre-trained model can be used to encode the input corpus to obtain the encoded input corpus. In one example, the trained pre-trained model can be a BERT model, which is a Transformer-based pre-trained language model that can semantically encode natural language to effectively extract semantic information from the input corpus.
[0049] The BERT model comes with multiple hidden layers. For example, the basic version of the BERT model BERT-Base can include 12 layers (Encoder layers), 12 self-attention heads (Attention heads) and 768 hidden layer sizes (Hidden size), and the enlarged version of the BERT model BERT-Large can include 24 layers (Encoder layers), 16 self-attention heads (Attention heads) and 1024 hidden layer sizes (Hidden size). In the embodiment of the present application, the hidden layer of the BERT model can be used to semantically encode the input corpus to obtain the encoded input corpus.
[0050] In an embodiment of the present application, N trained parallel hidden layers can also be used to extract features from the encoded input corpus to obtain N hierarchical semantic vectors. Among them, the N trained hidden layers are independent of the BERT model and are additionally set up. The number of hidden layers N is the same as the number of semantic levels, and the number of semantic layers can be set according to actual needs and the current business scenario. Furthermore, each of the N hierarchical semantic vectors represents the semantic information of the input corpus at a semantic level.
[0051] In the embodiment of the present application, N trained parallel hidden layers can convert the encoded input corpus output by the BERT model into semantic information vectors of each level of intent with a dimension of m, where m is a positive integer. That is, the input corpus after N trained parallel hidden layers is converted into N hierarchical semantic vectors, and the dimension of each hierarchical semantic vector is m. At this time, the first hierarchical semantic vector may include the hierarchical semantic feature L 11 To L 1m , the second hierarchical semantic vector can include the hierarchical semantic feature L 21 To L 2m , and so on, the Nth hierarchical semantic vector can include the hierarchical semantic feature L N1 To L Nm .
[0052] In the embodiment of the present application, the trained prediction model can be used to predict N hierarchical semantic vectors to obtain a target prediction result, wherein the target prediction result includes target hierarchical semantic features in at least two different hierarchical semantic vectors and the connection relationship between the target hierarchical semantic features.
[0053] In this way, the trained prediction model can predict semantic features including multiple different hierarchical semantic vectors, which can be some hierarchical semantic vectors or all hierarchical semantic vectors in N hierarchical semantic vectors. When the input corpus has matching intentions in each of the N semantic levels, the target prediction result includes the target hierarchical semantic features in all N hierarchical semantic vectors. It should be noted that each hierarchical semantic vector includes only one target hierarchical semantic feature. On the other hand, when the input corpus has matching intentions only in some of the N semantic levels, the target prediction result includes the target hierarchical semantic features in some of the N hierarchical semantic vectors that can match the input corpus. At this time, each hierarchical semantic vector of the partial hierarchical semantic vector also includes only one target hierarchical semantic feature.
[0054] In the embodiment of the present application, the intent of the input corpus can be determined using the hierarchical semantic features of each target and their connection relationship. In other words, the hierarchical semantic features of each target can be combined according to their connection relationship to obtain the hierarchical intent of the input corpus. Among them, the connection relationship of each hierarchical semantic feature of each target can also include a connection order, so when combining, it is necessary to combine each hierarchical semantic feature of each target according to the connection order.
[0055] According to the technical solution provided in the embodiment of the present application, the input corpus is first encoded, and then N trained parallel hidden layers are used to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors, and then the trained prediction model is used to predict the N hierarchical semantic vectors to obtain prediction results. Finally, the intention of the input corpus is determined according to the hierarchical semantic features of each target in the prediction results and their connection relationships. A single model is used to realize the recognition of multi-level semantic intentions, which reduces the system complexity, shortens the intention recognition process, improves the recognition efficiency, and enhances the user experience.
[0056] Figure 2 1 is a flow chart of a method for encoding input corpus to obtain encoded input corpus provided in an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:
[0057] In step S201, the pre-trained model is used to segment the input corpus to obtain the segmentation result of the input corpus.
[0058] In step S202, the word segmentation result is mapped to a preset word list to obtain a mapping result.
[0059] In step S203, the mapping result is encoded using the trained pre-trained model to obtain an encoded result.
[0060] In an embodiment of the present application, the trained pre-trained model may be a BERT model. When using the BERT model to encode an input corpus, the trained BERT model may be used to segment the input corpus to obtain a segmentation result of the input corpus, and then the segmentation result may be mapped to a preset vocabulary to obtain a mapping result, and finally the trained BERT model may be used to encode the segmentation result to obtain an encoding result.
[0061] Figure 3 1 is a flow chart of a method for extracting features from an encoded input corpus using N trained parallel hidden layers to obtain N hierarchical semantic vectors, as provided in an embodiment of the present application. Figure 3 As shown, the method comprises the following steps:
[0062] In step S301, the business scenario corresponding to the input corpus is obtained.
[0063] In step S302, the number of semantic levels N and the corresponding N trained hidden layers are determined based on the business scenario.
[0064] In step S303, the encoded input corpus is input into N trained hidden layers for feature extraction to obtain N hierarchical semantic vectors.
[0065] In the embodiment of the present application, when using N trained parallel hidden layers to extract features from the encoded input corpus, the business scenario corresponding to the input corpus can be first obtained, and the number of semantic levels N and the corresponding N trained hidden layers can be determined based on the business scenario. In other words, the number of semantic hierarchical layers can be determined according to the business scenario, and then the model structure corresponding to the number of semantic hierarchical layers, that is, the number of hidden layers, can be determined.
[0066] It should be noted that the operation steps of obtaining the business scenario to determine the number of semantic levels N and then determine the number of hidden layers N can also be performed before obtaining the input corpus, or after obtaining the input corpus and before performing the encoding operation, that is, different intent recognition global models can be pre-trained for different business scenarios. After the business scenario is determined, the intent recognition global model that matches the business scenario and has been trained, including a pre-trained model, N hidden layers and a prediction model, is called to perform intent recognition on the input corpus.
[0067] In the embodiment of the present application, after the number of hidden layers is determined, the encoded input corpus can be input into N trained hidden layers respectively for feature extraction to obtain N hierarchical semantic vectors.
[0068] Figure 4 1 is a flow chart of a method for predicting N hierarchical semantic vectors using a trained prediction model to obtain a target prediction result provided by an embodiment of the present application. Figure 4 As shown, the method comprises the following steps:
[0069] In step S401, N hierarchical semantic vectors are concatenated to obtain a global semantic vector.
[0070] In step S402, the global semantic vector is input into the trained fully connected layer to obtain the intention distribution probability of the input corpus at each semantic level.
[0071] In step S403, the intention distribution probability is predicted using the trained prediction model to obtain a target prediction result.
[0072] In an embodiment of the present application, the input corpus is processed by N hidden layers to obtain N hierarchical semantic vectors of dimension m. The N hierarchical semantic vectors of dimension m can be concatenated to obtain a global semantic vector of dimension N*m, and then the global semantic vector is input into a trained fully connected layer, which can be a fully connected layer for classification. Therefore, by inputting the global semantic vector into the fully connected layer, a preliminary probability distribution of each semantic intent level can be obtained.
[0073] In an embodiment of the present application, after obtaining the preliminary probability distribution of each semantic intent level, the trained prediction model can also be used to predict the intent distribution probability to obtain a target prediction result.
[0074] In the embodiment of the present application, the prediction model may be CRF. The CRF layer is a conditional random field model that calculates the probability of a path using the path, ie, the connection of the previous and next hierarchical semantic features, as a unit, and determines the path with the highest probability as the recommended path.
[0075] Figure 5 1 is a flow chart of a method for using a trained prediction model to predict the intention distribution probability and obtain a target prediction result provided by an embodiment of the present application. Figure 5 As shown, the method comprises the following steps:
[0076] In step S501, the intention distribution probability of the input corpus at each semantic level is input into the trained conditional random field to obtain the probability of the connection path between the first hierarchical semantic feature and the second hierarchical semantic feature.
[0077] The first hierarchical semantic feature and the second hierarchical semantic feature are two hierarchical semantic features in two adjacent semantic levels.
[0078] In step S502, the target path with the maximum probability is determined as the prediction result.
[0079] The target path includes at least M target hierarchical semantic features and the connection relationship between the M target hierarchical semantic features, where M is a positive integer greater than 1.
[0080] In the embodiment of the present application, after obtaining the preliminary probability distribution of each semantic intent level, the intent distribution probability of the input corpus at each semantic level can be input into the trained CRF to obtain the probability of the connection path between the first hierarchical semantic feature and the second hierarchical semantic feature, wherein the first hierarchical semantic feature and the second hierarchical semantic feature are two hierarchical semantic features in two adjacent semantic levels. In other words, after inputting the intent distribution probability of the input corpus at each semantic level into the trained CRF, the probability of the connection path between any hierarchical semantic feature and all hierarchical semantic features in its adjacent semantic levels can be obtained.
[0081] In an embodiment of the present application, the target path with the highest probability can be determined as the prediction result. Wherein, the target path includes at least M target hierarchical semantic features and the connection relationship between the M target hierarchical semantic features, and M is a positive integer greater than 1. In other words, the target path may include the target hierarchical semantic features in the hierarchical semantic vector corresponding to each semantic level in the N semantic levels, or may only include the target hierarchical semantic features in the hierarchical semantic vector corresponding to some semantic levels in the N semantic levels.
[0082] Figure 6 Schematic diagram of using CRF to predict the probability of intention distribution provided by the embodiment of the present application. Figure 6 As shown, the intention distribution probability includes the distribution probability of N semantic levels from left to right, and each semantic level may have n predicted probability labels, for example, there are n first-level labels label_1 to label_n at the first semantic level, n second-level labels label_1 to label_n at the second semantic level, and so on, there are n Nth-level labels label_1 to label_n at the Nth semantic level. Where n is a positive integer, and the values of labels label_1 to label_n at different semantic levels may be different.
[0083] Further, such as Figure 6As shown in the figure, the bold connecting lines indicate the connection paths between each level of intent. Based on context information, such as the semantic intent information of the upper and lower levels, the labels of each level and their adjacent labels can be scored separately, and the probability distribution of each semantic intent level can be further calculated, thereby improving the accuracy of intent recognition.
[0084] Figure 7 1 is a flow chart of a method for determining the intent of an input corpus using the hierarchical semantic features of each target and their connection relationships provided in an embodiment of the present application. Figure 7 As shown, the method comprises the following steps:
[0085] In step S701, label information of the semantic level corresponding to each target hierarchical semantic feature is obtained, and semantic information of each target semantic feature is determined based on the label information and the target hierarchical semantic feature.
[0086] In step S702, semantic information corresponding to each target hierarchical semantic feature is combined in sequence according to the connection relationship of each target hierarchical semantic feature to obtain the intention of the input corpus.
[0087] In an embodiment of the present application, when using each target hierarchical semantic feature and its connection relationship to determine the intention of the input corpus, the label information of the semantic level corresponding to each target hierarchical semantic feature can be first obtained, and the semantic information of each target semantic feature can be determined based on the label information and the target hierarchical semantic feature. Then, the semantic information corresponding to each target hierarchical semantic feature is sequentially combined according to the connection relationship of each target hierarchical semantic feature to obtain the intention of the input corpus.
[0088] Figure 8 Schematic diagram of the structure of the global model of intent recognition provided by the embodiment of the present application. Figure 8 As shown in the figure, the global model of intent recognition includes a BERT model, N parallel hidden layers, a concatenation unit, a fully connected layer, and a CRF. After the input corpus is encoded by the BERT model, it is sent to N parallel hidden layers for feature extraction to obtain N hierarchical semantic vectors. After the N hierarchical semantic vectors are concatenated in the concatenation unit, they are sent to the fully connected layer to obtain the intent distribution probability of the input corpus at each semantic level. CRF further performs path prediction on the intent distribution probability of the input corpus at each semantic level to obtain intent labels at each level.
[0089] Fig. 9 It is a flowchart of the training method of the global model of intent recognition provided in an embodiment of the present application.
[0090] like Fig. 9 As shown, the method comprises the following steps:
[0091] In step S901, training corpus is obtained.
[0092] In step S902, a classification label is added to the training corpus.
[0093] The hierarchical labels include labels for each of the N levels.
[0094] In step S903, the training corpus is encoded using the pre-training model to obtain the encoded training corpus.
[0095] In step S904, N parallel hidden layers are used to perform feature extraction on the encoded training corpus to obtain N training hierarchical semantic vectors.
[0096] Among them, each training graded semantic vector represents the semantic information of the training corpus at a semantic level.
[0097] In step S905, the prediction model is used to predict the N training hierarchical semantic vectors to obtain training prediction results.
[0098] In step S906, the training prediction result is compared with the classification label to obtain a loss function.
[0099] In step S907, in response to the loss function being greater than a preset threshold, at least one parameter in the pre-trained model, N hidden layers, and the prediction model is adjusted, and after the adjustment is completed, the steps of encoding the training corpus, performing feature extraction on the encoded training corpus to obtain N training graded semantic vectors, using the N training graded semantic vectors to perform predictions to obtain training prediction results, comparing the training prediction results with the graded labels to obtain the loss function, and adjusting the pre-trained model, N hidden layers, and at least one parameter in the prediction model when the loss function is greater than the preset threshold are repeated until the loss function is less than or equal to the preset threshold.
[0100] In step S908, in response to determining that the loss function is less than or equal to a preset threshold, the current pre-trained model, N hidden layers, and prediction model are determined to be a trained pre-trained model, N trained hidden layers, and a trained prediction model.
[0101] In the embodiment of the present application, when training the global model for intent recognition, the training corpus can be first obtained, and hierarchical labels can be added to the training corpus. Among them, the hierarchical labels include labels for each level in the N levels. In other words, human-computer interaction corpus can be collected, the level of interaction intent can be analyzed and set, and the corpus labels can be constructed based on the maximum level. When a certain level of label does not exist in some corpus, it can be filled with special marks such as the letter O.
[0102] In an embodiment of the present application, a pre-trained model can be used to encode the training corpus to obtain the encoded training corpus, and N parallel hidden layers can be used to perform feature extraction on the encoded training corpus to obtain N training hierarchical semantic vectors, each of which represents the semantic information of the training corpus at a semantic level.
[0103] In other words, the BERT model combined with N parallel hidden layers can be used to semantically encode the input natural language. The BERT model is a pre-trained language model based on Transformer, which can effectively extract the semantic information of the input text. The hidden layer is used to convert the output of the BERT model into a semantic information vector of each level of intent with a dimension of m.
[0104] In an embodiment of the present application, after being processed by N parallel hidden layers, the prediction model can be used to predict the N training hierarchical semantic vectors to obtain training prediction results. In one example, the N levels of hierarchical semantic vectors can be first spliced into N*m global semantic vectors, and then passed through a fully connected layer for classification to obtain the preliminary probability distribution of each semantic intent level. The CRF layer is connected after the fully connected layer. CRF is a conditional random field model that calculates the probability of considering the path in units of paths. Therefore, according to the context information, each level of labels and their adjacent labels can be scored separately, and the probability distribution of each semantic intent level can be further calculated, thereby improving the accuracy of intent recognition.
[0105] In an embodiment of the present application, after obtaining the training result, the training prediction result can be compared with the graded label to obtain a loss function. Then the loss function is compared with a preset threshold value. If the loss function is greater than the preset threshold value, the pre-trained model, N hidden layers, and at least one parameter in the prediction model are adjusted, and after the adjustment is completed, the steps of encoding the training corpus, extracting features from the encoded training corpus to obtain N training graded semantic vectors, using the N training graded semantic vectors to predict to obtain the training prediction result, comparing the training prediction result with the graded label to obtain the loss function, and adjusting the pre-trained model, N hidden layers, and at least one parameter in the prediction model when the loss function is greater than the preset threshold value are repeated until the loss function is less than or equal to the preset threshold value.
[0106] On the contrary, if the loss function is less than or equal to the preset threshold, the current pre-trained model, N hidden layers, and prediction model are determined to be the trained pre-trained model, N trained hidden layers, and trained prediction model.
[0107] In the training process, the maximum likelihood method can be used as the loss function. For example, the loss function can be -log P(y1, y2, ..., y n|x) means, where y1, y2, ..., y n is the semantic intent of level N, and x is the sample. In training, the gradient is back-propagated, and the weights of each category are adjusted by gradient descent. Because the loss is calculated based on the score of the label and the adjacent label, and there is a certain relationship between the labels at each level, for example, a label label_n will only appear in the second level and only after label_m, it can reduce the confusion between the multi-level semantic intent label classifications and improve the final recognition results.
[0108] Fig.10 FIG. 1 is a flow chart of another method for identifying intent provided in an embodiment of the present application. Fig.10 As shown in the figure, before performing intent recognition, we can first analyze the overall corpus data according to the usage scenario, determine the semantic classification level, and construct the classification label type. Among them, the semantic classification level needs to be determined by the largest number of levels in the corpus, that is, the most specific intent level.
[0109] In one example, taking the in-vehicle voice interaction usage scenario as an example, a total of 8 levels of hierarchical labels are constructed, and the type of hierarchical labels is: "domain-subdomain-intent-function-action-value-position-optional", where domain refers to the domain to which it belongs; subdomain refers to the subordinate domain; intent refers to the intention; function refers to the execution function; action refers to the execution operation; value refers to the operation value; position refers to the operation position; and optional refers to the operation option. For example, "turn on the air conditioner", domain is defined as "smart device", subdomain is defined as air conditioning equipment, intent is defined as state control, function is defined as air conditioning AC, action is defined as open, value is filled with the letter O if it does not exist in this example, position is defined as the air outlet position of the air conditioner, and optional is filled with the letter O if it does not exist in this example.
[0110] Next, the request text is constructed in the form of "[CLS] text [SEP]" and is segmented and encoded by Bert's tokenizer before being sent to the Bert model. Among them, [CLS] represents the start symbol and [SEP] represents the end symbol. After the text passes through the tokenizer, "[CLS], token0, token1, ..., [SEP]" is obtained, and then mapped into a sequence through the vocabulary, "101, x1, x2, ..., 102", "101" is [CLS], and "102" is [SEP]. Token0 and token1 are the segmentation results, and x1 and x2 are the mapping results.
[0111] More specifically, taking "turn on the air conditioner" as an example, the results obtained after the above two steps are: "[CLS], turn on, air conditioner, [SEP]", "101, *, 102". The "*" refers to the vocabulary mapping of the two tokens "turn on" and "air conditioner". In actual implementation, it is usually a specific number, which is represented by "*" here.
[0112] Next, the mapping result is encoded by BERT to obtain an overall semantic feature information, and then after N hidden layers are calculated, N levels of intent semantic information representation are obtained. Among them, the BERT model is a pre-trained language model based on Transformer, which can effectively extract the semantic information of the input text. Its output dimension is [batch_size, seq_len, hidden_size], batch_size is the batch size, without actual semantic information, and the value is 1 when a single sentence is input; hidden_size is the semantic information dimension, and the value is usually 768 in the BERT model; seq_len is the sequence length, which is the same as the mapping length and one-to-one corresponding. The vector corresponding to [CLS] can represent the overall semantic information of the sentence. Here, after the semantic vector is cut out (the vector dimension is [batch_size, hidden_size] at this time), it represents the semantic features of the entire sentence. The number of hidden layers N is determined by the hierarchical level.
[0113] More specifically, taking the in-vehicle voice interaction usage scenario as an example, as mentioned above, the hierarchical level can be defined as 8, and N is 8 at this time. That is, after obtaining the overall semantic information of the sentence, it is sent to 8 hidden layers respectively to obtain 8 hierarchical intent semantic information representations.
[0114] Next, the N semantic information representation vectors obtained can be concatenated and sent to the fully connected layer to calculate the probability distribution of intents at all levels. The concatenated semantic representation dimension is [batch_size, N, hidden_size], and the fully connected layer dimension is the sum of the number of intent labels at all levels M. After the fully connected layer, the probability distribution matrix [batch_size, N, M] is obtained. At this time, the specific value in the matrix represents the label probability corresponding to its index.
[0115] More specifically, taking "turn on the air conditioner" as an example, the number of intent levels in the in-vehicle voice interaction usage scenario is 8. Assuming that there are 500 intent labels at each level, the above M is 500 and N is 8.
[0116] Finally, the obtained probability distribution is input into the CRF layer to obtain the final label probability distribution. The label with the maximum probability is selected to obtain the final recognition result. Among them, the CRF layer is a conditional random field model that uses feature functions to define the transition probability between different labels. These feature functions can capture the dependency between labels, that is, according to the context information, such as the semantic intent information of the superior and the subordinate, the probability distribution of each semantic intent level is optimized, thereby improving the accuracy of the superior and subordinate intent recognition.
[0117] The technical solution of the embodiment of the present application combines the BERT model and CRF, which can understand the meaning of language more accurately and improve the accuracy of semantic understanding by relying on the correlation between multi-level intentions, so that the solution can not only be used for semantic understanding in human-computer interaction, but also can be widely used in information extraction, intelligent recommendation and other fields, and has broad application value.
[0118] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.
[0119] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.
[0120] Fig.11 Schematic diagram of an intention recognition device provided in an embodiment of the present application. Fig.11 As shown, the device comprises:
[0121] The encoding module 1101 is configured to obtain input corpus, encode the input corpus, and obtain encoded input corpus.
[0122] The feature extraction module 1102 is configured to use N trained parallel hidden layers to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors, where N is a positive integer, and each hierarchical semantic vector represents the semantic information of the input corpus at a semantic level.
[0123] The prediction module 1103 is configured to use the trained prediction model to predict N hierarchical semantic vectors to obtain a target prediction result, which includes target hierarchical semantic features in at least two different hierarchical semantic vectors and a connection relationship between each target hierarchical semantic feature.
[0124] The determination module 1104 is configured to determine the intention of the input corpus using each target hierarchical semantic feature and its connection relationship.
[0125] According to the technical solution provided in the embodiment of the present application, the input corpus is first encoded, and then N trained parallel hidden layers are used to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors, and then the trained prediction model is used to predict the N hierarchical semantic vectors to obtain prediction results. Finally, the intention of the input corpus is determined according to the hierarchical semantic features of each target in the prediction results and their connection relationships. A single model is used to realize the recognition of multi-level semantic intentions, which reduces the system complexity, shortens the intention recognition process, improves the recognition efficiency, and enhances the user experience.
[0126] In an embodiment of the present application, the input corpus is encoded to obtain the encoded input corpus, including: using a trained pre-trained model to segment the input corpus to obtain a segmentation result of the input corpus; mapping the segmentation result to a preset vocabulary to obtain a mapping result; and encoding the segmentation result using a trained pre-trained model to obtain an encoded result.
[0127] In an embodiment of the present application, N trained parallel hidden layers are used to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors, including: obtaining the business scenario corresponding to the input corpus; determining the number of semantic levels N and the corresponding N trained hidden layers based on the business scenario; and inputting the encoded input corpus into the N trained hidden layers for feature extraction to obtain N hierarchical semantic vectors.
[0128] In an embodiment of the present application, a trained prediction model is used to predict N hierarchical semantic vectors to obtain a target prediction result, including: concatenating the N hierarchical semantic vectors to obtain a global semantic vector; inputting the global semantic vector into a trained fully connected layer to obtain the intention distribution probability of the input corpus at each semantic level; and using the trained prediction model to predict the intention distribution probability to obtain a target prediction result.
[0129] In an embodiment of the present application, the trained prediction model is a trained conditional random field; the trained prediction model is used to predict the intention distribution probability to obtain a target prediction result, including: inputting the intention distribution probability of the input corpus at each semantic level into the trained conditional random field to obtain the probability of the connection path between the first hierarchical semantic feature and the second hierarchical semantic feature, wherein the first hierarchical semantic feature and the second hierarchical semantic feature are two hierarchical semantic features in two adjacent semantic levels; determining the target path with the maximum probability as the prediction result, wherein the target path includes at least M target hierarchical semantic features and the connection relationship between the M target hierarchical semantic features, and M is a positive integer greater than 1.
[0130] In an embodiment of the present application, the intention of the input corpus is determined using each target hierarchical semantic feature and their connection relationship, including: obtaining label information of the semantic level corresponding to each target hierarchical semantic feature, determining the semantic information of each target semantic feature based on the label information and the target hierarchical semantic feature; and combining the semantic information corresponding to each target hierarchical semantic feature according to the connection relationship sequence of each target hierarchical semantic feature to obtain the intention of the input corpus.
[0131] In the embodiment of the present application, the trained pre-trained model, N trained hidden layers, a trained fully connected layer and a trained prediction model are trained in the following manner: obtaining training corpus; adding hierarchical labels to the training corpus, the hierarchical labels including labels for each level of the N levels; encoding the training corpus using the pre-trained model to obtain encoded training corpus; performing feature extraction on the encoded training corpus using N parallel hidden layers to obtain N training hierarchical semantic vectors, each training hierarchical semantic vector representing the semantic information of the training corpus at a semantic level; predicting the N training hierarchical semantic vectors using the prediction model to obtain training prediction results; comparing the training prediction results with the hierarchical labels to obtain a loss function; in response to the loss function being greater than a preset threshold , adjusting at least one parameter of the pre-trained model, N hidden layers and the prediction model, and after the adjustment is completed, repeating the steps of encoding the training corpus, extracting features from the encoded training corpus to obtain N training graded semantic vectors, using the N training graded semantic vectors to perform predictions to obtain training prediction results, comparing the training prediction results with the graded labels to obtain a loss function, and adjusting the pre-trained model, N hidden layers and at least one parameter of the prediction model when the loss function is greater than a preset threshold, until the loss function is less than or equal to the preset threshold; in response to determining that the loss function is less than or equal to the preset threshold, determining that the current pre-trained model, N hidden layers and the prediction model are the trained pre-trained model, N trained hidden layers and the trained prediction model.
[0132] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0133] Fig.12 Schematic diagram of an electronic device provided in an embodiment of the present application. Fig.12 As shown, the electronic device 12 of this embodiment includes: a processor 1201, a memory 1202, and a computer program 1203 stored in the memory 1202 and executable on the processor 1201. When the processor 1201 executes the computer program 1203, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 1201 executes the computer program 1203, the functions of the modules / units in the above-mentioned device embodiments are implemented.
[0134] The electronic device 12 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 12 may include, but is not limited to, a processor 1201 and a memory 1202. Those skilled in the art will appreciate that Fig.12 The electronic device 12 is merely an example and does not limit the electronic device 12 , and may include more or fewer components than those shown in the figure, or different components.
[0135] Processor 1201 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0136] The memory 1202 may be an internal storage unit of the electronic device 12, for example, a hard disk or memory of the electronic device 12. The memory 1202 may also be an external storage device of the electronic device 12, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the electronic device 12. The memory 1202 may also include both an internal storage unit of the electronic device 12 and an external storage device. The memory 1202 is used to store computer programs and other programs and data required by the electronic device.
[0137] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.
[0138] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A method for intention recognition, characterized in that: include: Obtaining input corpus, and encoding the input corpus using a trained pre-trained model to obtain encoded input corpus; Using N trained parallel hidden layers to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors, where N is a positive integer, and each hierarchical semantic vector represents semantic information of the input corpus at a semantic level; Using the trained prediction model to predict the N hierarchical semantic vectors, obtaining a target prediction result, wherein the target prediction result includes target hierarchical semantic features in at least two different hierarchical semantic vectors, and a connection relationship between the target hierarchical semantic features; The intention of the input corpus is determined using the hierarchical semantic features of each target and their connection relationships.
2. The method according to claim 1, characterized in that The step of encoding the input corpus to obtain the encoded input corpus includes: Using the trained pre-trained model to segment the input corpus to obtain a segmentation result of the input corpus; Map the word segmentation results to the preset word list to obtain the mapping results; The mapping result is encoded using the trained pre-trained model to obtain the encoded result.
3. The method according to claim 1, characterized in that The method uses N trained parallel hidden layers to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors, including: Obtaining the business scenario corresponding to the input corpus; Determine the number of semantic levels N and the corresponding N trained hidden layers based on the business scenario; The encoded input corpus is respectively input into the N trained hidden layers for feature extraction to obtain N hierarchical semantic vectors.
4. The method according to claim 1, characterized in that: The using the trained prediction model to predict the N hierarchical semantic vectors to obtain a target prediction result includes: Concatenating the N hierarchical semantic vectors to obtain a global semantic vector; Input the global semantic vector into the trained fully connected layer to obtain the intention distribution probability of the input corpus at each semantic level; The intention distribution probability is predicted using the trained prediction model to obtain the target prediction result.
5. The method according to claim 4, characterized in that The trained prediction model is a trained conditional random field; The using the trained prediction model to predict the intention distribution probability to obtain the target prediction result includes: Inputting the intention distribution probability of the input corpus at each semantic level into the trained conditional random field to obtain the probability of a connection path between a first hierarchical semantic feature and a second hierarchical semantic feature, wherein the first hierarchical semantic feature and the second hierarchical semantic feature are two hierarchical semantic features in two adjacent semantic levels; A target path with a maximum probability is determined as the prediction result, wherein the target path includes at least M target hierarchical semantic features and a connection relationship between the M target hierarchical semantic features, and M is a positive integer greater than 1.
6. The method according to claim 1, characterized in that The method of using the hierarchical semantic features of each target and their connection relationships to determine the intention of the input corpus includes: Acquire label information of the semantic level corresponding to each target hierarchical semantic feature, and determine the semantic information of each target semantic feature based on the label information and the target hierarchical semantic feature; The semantic information corresponding to each target hierarchical semantic feature is combined in sequence according to the connection relationship of each target hierarchical semantic feature to obtain the intention of the input corpus.
7. The method according to any one of claims 1 to 6, characterized in that The trained pre-trained model, N trained hidden layers, trained fully connected layers, and trained prediction model are trained in the following way: Get training corpus; Adding hierarchical labels to the training corpus, wherein the hierarchical labels include labels for each level in N levels; Encoding the training corpus using a pre-training model to obtain encoded training corpus; Using N parallel hidden layers to perform feature extraction on the encoded training corpus to obtain N training hierarchical semantic vectors, each training hierarchical semantic vector representing semantic information of the training corpus at a semantic level; Use the prediction model to predict the N training hierarchical semantic vectors to obtain a training prediction result; Comparing the training prediction result with the classification label to obtain a loss function; In response to the loss function being greater than a preset threshold, adjusting at least one parameter of the pre-trained model, the N hidden layers, and the prediction model, and after the adjustment is completed, repeating the steps of encoding the training corpus, extracting features from the encoded training corpus to obtain N training graded semantic vectors, using the N training graded semantic vectors to perform predictions to obtain training prediction results, comparing the training prediction results with the graded labels to obtain the loss function, and adjusting at least one parameter of the pre-trained model, the N hidden layers, and the prediction model when the loss function is greater than the preset threshold, until the loss function is less than or equal to the preset threshold; In response to determining that the loss function is less than or equal to the preset threshold, the current pre-trained model, N hidden layers, and prediction model are determined to be the trained pre-trained model, N trained hidden layers, and trained prediction model.
8. An intention recognition device, characterized in that: include: An encoding module is configured to obtain input corpus, encode the input corpus, and obtain encoded input corpus; A feature extraction module is configured to use N trained parallel hidden layers to perform feature extraction on the encoded input corpus to obtain N hierarchical semantic vectors, where N is a positive integer, and each hierarchical semantic vector represents semantic information of the input corpus at a semantic level; A prediction module is configured to use the trained prediction model to predict the N hierarchical semantic vectors to obtain a target prediction result, wherein the target prediction result includes target hierarchical semantic features in at least two different hierarchical semantic vectors and a connection relationship between the target hierarchical semantic features; The determination module is configured to determine the intention of the input corpus using each target hierarchical semantic feature and its connection relationship.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Voice-driven interface interaction system and method based on task-based dialogue
CN121092108A
