Method and device for generating intention recognition processing and confidence judgment model

Abstract

The present application proposes a method and device for generating an intent recognition processing and confidence judgment model, wherein the intent recognition processing method includes: inputting the text to be recognized into the intent recognition model to obtain the intent recognition result; obtaining the final result features and intermediate result features of the intent recognition model, wherein the final result features are generated according to the intent recognition result; obtaining the sentence structure features of the text to be recognized; inputting the final result features, intermediate result features and sentence structure features into the confidence judgment model to obtain the confidence result corresponding to the intent recognition result. The intent recognition processing method proposed in the present application combines the features of the final result features, the intermediate result features and the sentence structure features to judge whether the intent recognition result is credible, thereby improving the accuracy and reducing the error.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method and device for generating an intent recognition processing and confidence judgment model. Background Art

[0002] At present, the intelligent outbound call system, as an important application of artificial intelligence, has appeared in all aspects of our lives, such as intelligent return visits, intelligent appointments, etc. In the intelligent outbound call system, judging the user's intention is a very important step, which is related to each step of the system process.

[0003] In related technologies, the credibility of intent recognition results is often determined based on the output results of the machine learning or deep learning model itself, which has low accuracy and large errors. Summary of the invention

[0004] The present application proposes a method and device for generating an intent recognition processing and confidence judgment model.

[0005] The first aspect of the present application proposes an intention recognition processing method, including: inputting the text to be recognized into an intention recognition model to obtain an intention recognition result; obtaining the final result features and intermediate result features of the intention recognition model, wherein the final result features are generated according to the intention recognition result; obtaining the sentence structure features of the text to be recognized; inputting the final result features, the intermediate result features and the sentence structure features into a confidence judgment model to obtain a confidence result corresponding to the intention recognition result.

[0006] The intent recognition processing method of the embodiment of the present application inputs the text to be recognized into the intent recognition model to obtain the intent recognition result, obtains the final result features and intermediate result features of the intent recognition model, wherein the final result features are generated according to the intent recognition result, obtains the sentence structure features of the text to be recognized, and inputs the final result features, intermediate result features and sentence structure features into the confidence judgment model to obtain the confidence result corresponding to the intent recognition result. In this embodiment, the text to be recognized is input into the intent recognition model to obtain the final result features and intermediate result features, and together with the sentence structure features, three aspects of features are input into the confidence judgment model to obtain the confidence result of the intent recognition result. Since the three aspects of features are combined to judge whether the intent recognition result is credible, the accuracy is improved and the error is reduced.

[0007] The second aspect of the present application proposes a method for generating a confidence judgment model, including: inputting a sample text into an intent recognition model to obtain a sample intent recognition result; obtaining a sample final result feature and a sample intermediate result feature of the intent recognition model, wherein the sample final result feature is generated according to the sample intent recognition result; obtaining a sample sentence structure feature of the sample text; inputting the sample final result feature, the sample intermediate result feature and the sample sentence structure feature into a confidence judgment model to be trained to obtain a sample confidence result corresponding to the sample intent recognition result; training the parameters of the confidence judgment model to be trained according to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text to obtain the confidence judgment model.

[0008] The method for generating a confidence judgment model in an embodiment of the present application inputs a sample text into an intent recognition model to obtain a sample intent recognition result, obtains a sample final result feature and a sample intermediate result feature of the intent recognition model, wherein the sample final result feature is generated according to the sample intent recognition result, obtains a sample sentence structure feature of the sample text, inputs the sample final result feature, the sample intermediate result feature and the sample sentence structure feature into a confidence judgment model to be trained to obtain a sample confidence result corresponding to the sample intent recognition result, and trains the parameters of the confidence judgment model to be trained according to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text to obtain a confidence judgment model. In this embodiment, the sample text is input into the intent recognition model to obtain the sample final result features and the sample intermediate result features, and together with the sample sentence structure features, three aspects of features are input into the confidence judgment model to be trained to obtain the sample confidence result of the sample intent recognition result, and the parameters of the confidence judgment model to be trained are trained in combination with the actual intent recognition result of the sample text to obtain the confidence judgment model. Since the three aspects of features are combined to judge whether the sample intent recognition result is credible, the accuracy is improved and the error is reduced.

[0009] The third aspect embodiment of the present application proposes an intention recognition processing device, comprising: a first input module, configured to input the text to be recognized into an intention recognition model to obtain an intention recognition result; a first acquisition module, configured to obtain the final result features and intermediate result features of the intention recognition model, wherein the final result features are generated according to the intention recognition result; a second acquisition module, configured to obtain the sentence structure features of the text to be recognized; and a second input module, configured to input the final result features, the intermediate result features and the sentence structure features into a confidence judgment model to obtain a confidence result corresponding to the intention recognition result.

[0010] The fourth aspect of the present application proposes a device for generating a confidence judgment model, comprising: a third input module, configured to input a sample text into an intent recognition model to obtain a sample intent recognition result; a third acquisition module, configured to obtain sample final result features and sample intermediate result features of the intent recognition model, wherein the sample final result features are generated according to the sample intent recognition result; a fourth acquisition module, configured to obtain sample sentence structure features of the sample text; a fourth input module, configured to input the sample final result features, the sample intermediate result features and the sample sentence structure features into a confidence judgment model to be trained to obtain a sample confidence result corresponding to the sample intent recognition result; a training module, configured to train the parameters of the confidence judgment model to be trained according to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text to obtain the confidence judgment model.

[0011] The fifth aspect embodiment of the present application proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the intention recognition processing method as described in the first aspect embodiment above, or execute the confidence judgment model generation method as described in the second aspect embodiment above.

[0012] The sixth aspect embodiment of the present application proposes a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the intention recognition processing method as described in the first aspect embodiment above, or to execute the confidence judgment model generation method as described in the second aspect embodiment above.

[0013] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0015] Figure 1 A flowchart of an intent recognition processing method provided in one embodiment of the present application;

[0016] Figure 2 A schematic diagram of the characteristic components of the intent recognition processing method provided in one embodiment of the present application;

[0017] Figure 3A flowchart of an intent recognition processing method provided by another embodiment of the present application;

[0018] Figure 4 A schematic diagram of a flow chart of a method for generating a confidence judgment model provided in an embodiment of the present application;

[0019] Figure 5 A schematic diagram of a flow chart of a method for generating a confidence judgment model provided in another embodiment of the present application;

[0020] Figure 6 A schematic diagram of the overall process of a method for generating an intention recognition process and a confidence judgment model provided in another embodiment of the present application;

[0021] Figure 7 A schematic diagram of the structure of an intention recognition processing device provided in one embodiment of the present application;

[0022] Figure 8 A schematic diagram of the structure of a device for generating a confidence judgment model provided in an embodiment of the present application;

[0023] Fig. 9 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0024] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0025] The following describes the intent recognition processing and confidence judgment model generation method and device of the embodiment of the present application with reference to the accompanying drawings.

[0026] Figure 1 The flowchart of the intention recognition processing method provided in one embodiment of the present application is shown in FIG. The intention recognition processing method of the embodiment of the present application can be executed by the intention recognition processing device provided in the embodiment of the present application, and the intention recognition processing device can be set in the intelligent outbound call system. Figure 1 As shown, the intention recognition processing method of the embodiment of the present application may specifically include the following steps:

[0027] S101, inputting the text to be recognized into the intent recognition model to obtain the intent recognition result.

[0028] Before constructing the intent recognition model, the intent recognition processing method of the embodiment of the present application must first determine the intent category, that is, have a clear classification and definition of the intent, and then determine the intent recognition model, such as the logistic regression model in the machine learning model, the text classification convolutional neural network (TextConvolutional Neural Networks, TextCNN) model in the deep learning model, etc.

[0029] Specifically, the text to be recognized is the text of the current intent to be recognized. The text to be recognized is input into a pre-built intent recognition model, and the intent recognition model processes the text to be recognized through natural language processing (NLP) to obtain an intent recognition result. The intent recognition result may specifically include but is not limited to various intents (such as Figure 1 ,meaning Figure 2 ,meaning Figure 3 ) and the probability corresponding to each type of intent, such as Figure 1 Harmony Figure 1 The corresponding probability is 0.6, Figure 2 Harmony Figure 2 The corresponding probability is 0.3, Figure 3 Harmony Figure 3 The corresponding probability is 0.1.

[0030] S102, obtaining final result features and intermediate result features of the intent recognition model, wherein the final result features are generated according to the intent recognition results.

[0031] Specifically, the final result features and intermediate result features of the intent recognition model are obtained, and the final result features are generated according to the intent recognition result obtained in step S101. The final result features may specifically include but are not limited to at least one of the following: the intent recognition vector corresponding to the intent recognition result, the highest score top1 corresponding to the intent recognition result, and the difference between the highest score top1 and the second highest score top2 corresponding to the intent recognition result (top1-top2), etc. The intermediate result features may specifically include but are not limited to the vector output by the hidden layer of the intent recognition model, etc.

[0032] S103, obtaining sentence structure features of the text to be recognized.

[0033] Specifically, the sentence structure features of the text to be recognized input in step S101 are obtained. The sentence structure features may specifically include but are not limited to at least one of the following: sentence length, sentence perplexity, sentence analysis results, and the like.

[0034] S104, inputting the final result features, the intermediate result features and the sentence structure features into the confidence judgment model to obtain the confidence result corresponding to the intent recognition result.

[0035] Specifically, the final result features, intermediate result features and sentence structure features obtained in step S102 are concatenated and input into the trained confidence judgment model. The confidence judgment model outputs the confidence result corresponding to the intention recognition result, that is, whether the intention recognition result obtained in step S101 is credible. The schematic diagram of the feature composition input into the confidence judgment model is shown in FIG. Figure 2 The confidence judgment model can be obtained by training the confidence judgment model generation method of the following embodiment.

[0036] The intent recognition processing method of the embodiment of the present application inputs the text to be recognized into the intent recognition model to obtain the intent recognition result, obtains the final result features and intermediate result features of the intent recognition model, wherein the final result features are generated according to the intent recognition result, obtains the sentence structure features of the text to be recognized, and inputs the final result features, intermediate result features and sentence structure features into the confidence judgment model to obtain the confidence result corresponding to the intent recognition result. In this embodiment, the text to be recognized is input into the intent recognition model to obtain the final result features and intermediate result features, and together with the sentence structure features, three aspects of features are input into the confidence judgment model to obtain the confidence result of the intent recognition result. Since the three aspects of features are combined to judge whether the intent recognition result is credible, the accuracy is improved and the error is reduced.

[0037] Figure 3 This is a flow chart of an intention recognition processing method provided by another embodiment of the present application. Figure 3 As shown in the above Figure 1 Based on the illustrated embodiment, the intention recognition processing method of the embodiment of the present application may specifically include the following steps:

[0038] S301, outputting a first inquiry voice.

[0039] Specifically, in the embodiments of the present application, Figure 1 The intention recognition processing method of the illustrated embodiment is applied to an intelligent outbound calling system. The intelligent outbound calling system can output a first inquiry voice through a voice control device such as a telephone.

[0040] S302: Obtain the user's answer voice to the first inquiry voice.

[0041] Specifically, the answer voice of the user to the first inquiry voice output in step S301 is obtained. The answer voice may specifically include but is not limited to the voice stream spoken by the user over the phone.

[0042] S203, generating a text to be recognized according to the answering voice.

[0043] Specifically, the intelligent outbound call system converts the answer voice obtained in step S303 into text through automatic speech recognition technology (Automatic Speech Recognition, ASR) to generate text to be recognized.

[0044] S304: input the text to be recognized into the intent recognition model to obtain the intent recognition result.

[0045] S305, obtaining final result features and intermediate result features of the intent recognition model, wherein the final result features are generated according to the intent recognition results.

[0046] S306: Acquire sentence structure features of the text to be recognized.

[0047] S307, input the final result features, the intermediate result features and the sentence structure features into the confidence judgment model to obtain the confidence result corresponding to the intent recognition result.

[0048] Specifically, steps S304-S307 in the embodiment of the present application are the same as steps S101-S104 in the above embodiment, and will not be repeated here.

[0049] As a feasible implementation method, the intention recognition processing method provided in the embodiment of the present application may also include the following steps:

[0050] S308: If the confidence result is yes, the intention recognition result is confirmed as the target intention recognition result, and a second inquiry voice is output according to the target intention recognition result.

[0051] Specifically, if the confidence result obtained in step S307 is yes, the intention recognition result obtained in step S304 is credible, the intention recognition result is confirmed as the target intention recognition result, and the next inquiry voice, i.e., the second inquiry voice, is output according to the target intention recognition result to proceed with the subsequent process.

[0052] S309: If the confidence result is no, the first inquiry voice is output again.

[0053] Specifically, if the confidence result obtained in step S307 is negative, the intention recognition result obtained in step S304 is not credible, and the first inquiry voice is re-output and the process is repeated. In the embodiment of the present application, the same process can be set to be repeated up to 3 times to avoid affecting the user experience.

[0054] The intention recognition processing method of the embodiment of the present application inputs the text to be recognized into the intention recognition model to obtain the intention recognition result, obtains the final result feature and the intermediate result feature of the intention recognition model, wherein the final result feature is generated according to the intention recognition result, obtains the sentence structure feature of the text to be recognized, and inputs the final result feature, the intermediate result feature and the sentence structure feature into the confidence judgment model to obtain the confidence result corresponding to the intention recognition result. In this embodiment, the user's answer voice is converted into the text to be recognized by automatic speech recognition technology ASR, and the text to be recognized is input into the intention recognition model to obtain the final result feature and the intermediate result feature, and together with the sentence structure feature, the three aspects of features are input into the confidence judgment model to obtain the confidence result of the intention recognition result. Since the three aspects of features are combined to judge whether the intention recognition result is credible, the accuracy is improved and the error is reduced. By re-asking the text to be recognized with a low confidence in intention recognition and re-determining the intention recognition result, the accuracy is further improved and the error is reduced.

[0055] Figure 4 The flowchart of the method for generating a confidence judgment model provided in one embodiment of the present application is shown in FIG. The method for generating a confidence judgment model in the embodiment of the present application can be executed by the device for generating a confidence judgment model provided in the embodiment of the present application, and the device for generating a confidence judgment model can be set in an intelligent outbound calling system. Figure 4 As shown, the method for generating the confidence judgment model in the embodiment of the present application may specifically include the following steps:

[0056] S401, inputting the sample text into the intent recognition model to obtain the sample intent recognition result.

[0057] Specifically, the sample text is input into a pre-built intent recognition model, and the intent recognition model processes the sample text through natural language processing (NLP) to obtain a sample intent recognition result. In an embodiment of the present application, after intent recognition, the actual intent of the sample text is manually annotated to obtain an actual intent recognition result corresponding to the sample text.

[0058] S402, obtaining sample final result features and sample intermediate result features of the intent recognition model, wherein the sample final result features are generated according to the sample intent recognition results.

[0059] S403, obtaining sample sentence structure features of the sample text.

[0060] S404, inputting the sample final result features, the sample intermediate result features and the sample sentence structure features into the confidence judgment model to be trained to obtain the sample confidence result corresponding to the sample intent recognition result.

[0061] Specifically, steps S402-S404 in the embodiment of the present application are similar to steps S102-S104 in the above embodiment and are not described again here.

[0062] S405 , training the parameters of the confidence judgment model to be trained according to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text, to obtain the confidence judgment model.

[0063] Specifically, according to the sample confidence result obtained in step S404, the sample intent recognition result obtained in step S401 and the actual intent recognition result corresponding to the sample text, the parameters of the confidence judgment model to be trained are trained to obtain the confidence judgment model. The confidence judgment model may specifically include but is not limited to support vector machine (Support Vector Regression, referred to as SVR), extreme gradient boosting (eXtreme Gradient Boosting, referred to as XGBoost), etc.

[0064] In an embodiment of the present application, if the sample intent recognition result is consistent with the actual intent recognition result, the sample text and the actual intent recognition result constitute a negative sample, which corresponds to (0, 1). For example, if the sample confidence result is (0.6, 0.4), then the probability that the sample confidence result is 0 (no) is 0.6, and the probability that the sample confidence result is 1 (yes) is 0.4. The loss is calculated based on 0.4 and 1, and the parameters of the confidence judgment model are adjusted based on the loss. If the sample intent recognition result is inconsistent with the actual intent recognition result, the sample text and the actual intent recognition result constitute a positive sample, which corresponds to (1, 0). For example, if the sample confidence result is (0.6, 0.4), then the probability that the sample confidence result is 1 (yes) is 0.6, and the probability that the sample confidence result is 0 (no) is 0.4. The loss is calculated based on 0.4 and 0, and the parameters of the confidence judgment model are adjusted based on the loss.

[0065] The method for generating a confidence judgment model in an embodiment of the present application inputs a sample text into an intent recognition model to obtain a sample intent recognition result, obtains a sample final result feature and a sample intermediate result feature of the intent recognition model, wherein the sample final result feature is generated according to the sample intent recognition result, obtains a sample sentence structure feature of the sample text, inputs the sample final result feature, the sample intermediate result feature and the sample sentence structure feature into a confidence judgment model to be trained to obtain a sample confidence result corresponding to the sample intent recognition result, and trains the parameters of the confidence judgment model to be trained according to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text to obtain a confidence judgment model. In this embodiment, the sample text is input into the intent recognition model to obtain the sample final result features and the sample intermediate result features, and together with the sample sentence structure features, three aspects of features are input into the confidence judgment model to be trained to obtain the sample confidence result of the sample intent recognition result, and the parameters of the confidence judgment model to be trained are trained in combination with the actual intent recognition result of the sample text to obtain the confidence judgment model. Since the three aspects of features are combined to judge whether the sample intent recognition result is credible, the accuracy is improved and the error is reduced.

[0066] Figure 5 A flow chart of a method for generating a confidence judgment model provided in another embodiment of the present application. Figure 5 As shown in the above Figure 4 Based on the illustrated embodiment, the method for generating a confidence judgment model in the embodiment of the present application may specifically include the following steps:

[0067] S501, output sample inquiry speech.

[0068] S502: Obtain a sample answer voice of the user in response to a sample inquiry voice.

[0069] S503, generating a sample text according to the sample answer voice.

[0070] Specifically, steps S501-S503 in the embodiment of the present application are similar to steps S301-S303 in the above embodiment and are not described again here.

[0071] S504: Input the sample text into the intent recognition model to obtain a sample intent recognition result.

[0072] S505, obtaining sample final result features and sample intermediate result features of the intent recognition model, wherein the sample final result features are generated according to the sample intent recognition results.

[0073] S506, obtaining sample sentence structure features of the sample text.

[0074] S507, input the sample final result features, the sample intermediate result features and the sample sentence structure features into the confidence judgment model to be trained to obtain the sample confidence result corresponding to the sample intent recognition result.

[0075] S508, training the parameters of the confidence judgment model to be trained according to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text, to obtain the confidence judgment model.

[0076] Specifically, steps S505-S508 in the embodiment of the present application are similar to steps S401-S405 in the above embodiment and are not described again here.

[0077] The method for generating a confidence judgment model in an embodiment of the present application inputs a sample text into an intent recognition model to obtain a sample intent recognition result, obtains a sample final result feature and a sample intermediate result feature of the intent recognition model, wherein the sample final result feature is generated according to the sample intent recognition result, obtains a sample sentence structure feature of the sample text, inputs the sample final result feature, the sample intermediate result feature and the sample sentence structure feature into a confidence judgment model to be trained to obtain a sample confidence result corresponding to the sample intent recognition result, and trains the parameters of the confidence judgment model to be trained according to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text to obtain a confidence judgment model. In this embodiment, before constructing the confidence judgment model, the user sample answer voice is converted into sample text through automatic speech recognition technology ASR to provide training data, and the sample text is input into the intent recognition model to obtain the sample final result features and the sample intermediate result features, and together with the sample sentence structure features, three aspects of features are input into the confidence judgment model to be trained to obtain the sample confidence result of the sample intent recognition result, and the parameters of the confidence judgment model to be trained are trained in combination with the actual intent recognition results of the sample text to obtain the confidence judgment model. Since the three aspects of features are combined to judge whether the sample intent recognition result is credible, the accuracy is improved and the error is reduced.

[0078] Figure 6 FIG. 1 is an overall flow chart of the method for generating the intention recognition processing and confidence judgment model of the embodiment of the present application. Figure 6 As shown, the method for generating the intention recognition processing and confidence judgment model in the embodiment of the present application may specifically include the following steps:

[0079] S601, output sample inquiry speech.

[0080] S602, obtaining a sample answer voice of the user in response to the sample inquiry voice.

[0081] S603, generating a sample text according to the sample answer voice.

[0082] S604: Input the sample text into the intent recognition model to obtain a sample intent recognition result.

[0083] S605, obtaining sample final result features and sample intermediate result features of the intent recognition model.

[0084] S606, obtaining sample sentence structure features of the sample text.

[0085] S607, input the sample final result features, the sample intermediate result features and the sample sentence structure features into the confidence judgment model to be trained to obtain the sample confidence result corresponding to the sample intent recognition result.

[0086] S608: According to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text, the parameters of the confidence judgment model to be trained are trained to obtain the confidence judgment model

[0087] S609, output the first inquiry voice

[0088] S610: Obtain a user's answer voice to a first inquiry voice.

[0089] S611, generating text to be recognized according to the answering voice.

[0090] S612: Input the text to be recognized into the intent recognition model to obtain the intent recognition result.

[0091] S613, obtaining final result features and intermediate result features of the intent recognition model, wherein the final result features are generated according to the intent recognition results.

[0092] S614, obtaining sentence structure features of the text to be recognized.

[0093] S615, input the final result features, the intermediate result features and the sentence structure features into the confidence judgment model to obtain the confidence result corresponding to the intent recognition result.

[0094] If the confidence result is yes, execute step S616; if the confidence result is no, execute step S609.

[0095] S616: Confirm the intention recognition result as the target intention recognition result, and output a second inquiry voice according to the target intention recognition result.

[0096] In order to implement the above embodiment, the embodiment of the present application also proposes an intent recognition processing device. Figure 7 FIG. 1 is a schematic diagram of the structure of an intention recognition processing device according to an embodiment of the present application. Figure 7As shown, the intention recognition processing device 700 of the embodiment of the present application may specifically include: a first input module 701, a first acquisition module 702, a second acquisition module 703 and a second input module 704.

[0097] The first input module 701 is configured to input the text to be recognized into the intention recognition model to obtain the intention recognition result.

[0098] The first acquisition module 702 is configured to acquire the final result features and intermediate result features of the intent recognition model, wherein the final result features are generated according to the intent recognition results.

[0099] The second acquisition module 703 is configured to acquire sentence structure features of the text to be recognized.

[0100] The second input module 704 is configured to input the final result features, the intermediate result features and the sentence structure features into the confidence judgment model to obtain a confidence result corresponding to the intent recognition result.

[0101] In one embodiment of the present application, the final result features include at least one of the following: an intent recognition vector corresponding to the intent recognition result, the highest score corresponding to the intent recognition result, and the difference between the highest score and the second highest score corresponding to the intent recognition result.

[0102] In one embodiment of the present application, the intermediate result features include a vector output by a hidden layer of the intent recognition model.

[0103] In one embodiment of the present application, the sentence structure feature includes at least one of the following: sentence length, sentence perplexity, and sentence analysis result.

[0104] In one embodiment of the present application, the intention recognition processing device 700 also includes: a first output module 705, configured to output a first inquiry voice; a fifth acquisition module 706, configured to obtain the user's answer voice to the first inquiry voice; and a first generation module 707, configured to generate a text to be recognized based on the answer voice.

[0105] In one embodiment of the present application, the intention recognition processing device 700 also includes: a second output module 708, which is configured to confirm the intention recognition result as a target intention recognition result if the confidence result is yes, and output a second inquiry voice according to the target intention recognition result; a third output module 709, which is configured to re-output the first inquiry voice if the confidence result is no.

[0106] It should be noted that the above explanation of the embodiment of the intent recognition processing method is also applicable to the intent recognition processing device of the embodiment of the present disclosure, and the specific process will not be repeated here.

[0107] The intention recognition processing device of the embodiment of the present application inputs the text to be recognized into the intention recognition model to obtain the intention recognition result, obtains the final result feature and the intermediate result feature of the intention recognition model, wherein the final result feature is generated according to the intention recognition result, obtains the sentence structure feature of the text to be recognized, and inputs the final result feature, the intermediate result feature and the sentence structure feature into the confidence judgment model to obtain the confidence result corresponding to the intention recognition result. In this embodiment, the text to be recognized is input into the intention recognition model to obtain the final result feature and the intermediate result feature, and together with the sentence structure feature, three aspects of features are input into the confidence judgment model to obtain the confidence result of the intention recognition result. Since the three aspects of features are combined to judge whether the intention recognition result is credible, the accuracy is improved and the error is reduced.

[0108] In order to implement the above embodiment, the embodiment of the present application also proposes a device for generating a confidence judgment model. Figure 8 FIG. 1 is a schematic diagram of a structure of a device for generating a confidence judgment model according to an embodiment of the present application. Figure 8 As shown, the confidence judgment model generation device 800 of the embodiment of the present application may specifically include: a third input module 801, a third acquisition module 802, a fourth acquisition module 803, a fourth input module 804 and a training module 805.

[0109] The third input module 801 is configured to input the sample text into the intent recognition model to obtain a sample intent recognition result.

[0110] The third acquisition module 802 is configured to acquire sample final result features and sample intermediate result features of the intent recognition model, wherein the sample final result features are generated according to the sample intent recognition results.

[0111] The fourth acquisition module 803 is configured to acquire sample sentence structure features of the sample text.

[0112] The fourth input module 804 is configured to input the sample final result features, the sample intermediate result features and the sample sentence structure features into the confidence judgment model to be trained to obtain a sample confidence result corresponding to the sample intent recognition result.

[0113] The training module 805 is configured to train the parameters of the confidence judgment model to be trained according to the sample confidence results, the sample intent recognition results and the actual intent recognition results corresponding to the sample text to obtain the confidence judgment model.

[0114] In one embodiment of the present application, the sample final result features include at least one of the following: an intent recognition vector corresponding to the sample intent recognition result, the highest score corresponding to the sample intent recognition result, and the difference between the highest score and the second highest score corresponding to the sample intent recognition result.

[0115] In one embodiment of the present application, the sample intermediate result feature includes a vector output by a hidden layer of the intent recognition model.

[0116] In one embodiment of the present application, the sample sentence structure features include at least one of the following: sentence length, sentence perplexity, and sentence analysis results.

[0117] In one embodiment of the present application, the confidence judgment model generation device 800 also includes: a fourth output module 806, configured to output a sample inquiry voice; a sixth acquisition module 807, configured to obtain a sample answer voice of the user to the sample inquiry voice; and a second generation module, configured to generate a sample text based on the sample answer voice.

[0118] It should be noted that the above explanation of the embodiment of the method for generating a confidence judgment model is also applicable to the device for generating a confidence judgment model in the embodiment of the present disclosure, and the specific process will not be repeated here.

[0119] The confidence judgment model generation device of the embodiment of the present application inputs a sample text into an intent recognition model to obtain a sample intent recognition result, obtains a sample final result feature and a sample intermediate result feature of the intent recognition model, wherein the sample final result feature is generated according to the sample intent recognition result, obtains a sample sentence structure feature of the sample text, inputs the sample final result feature, the sample intermediate result feature and the sample sentence structure feature into the confidence judgment model to be trained to obtain a sample confidence result corresponding to the sample intent recognition result, and trains the parameters of the confidence judgment model to be trained according to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text to obtain a confidence judgment model. In this embodiment, the sample text is input into the intent recognition model to obtain the sample final result features and the sample intermediate result features, and together with the sample sentence structure features, three aspects of features are input into the confidence judgment model to be trained to obtain the sample confidence result of the sample intent recognition result, and the parameters of the confidence judgment model to be trained are trained in combination with the actual intent recognition result of the sample text to obtain the confidence judgment model. Since the three aspects of features are combined to judge whether the sample intent recognition result is credible, the accuracy is improved and the error is reduced.

[0120] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0121] like Fig. 9, is a block diagram of an electronic device according to a file conversion method according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as intelligent voice interaction devices, personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present application described herein and / or required.

[0122] like Fig. 9 As shown, the electronic device includes: one or more processors 901, a memory 902, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor 901 can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 901 is taken as an example.

[0123] The memory 902 is a non-transitory computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to perform the intent recognition processing and confidence judgment model generation method provided in the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the intent recognition processing and confidence judgment model generation method provided in the present application.

[0124] The memory 902 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the file conversion method in the embodiment of the present application (for example, the attached Figure 7 The first input module 701, the first acquisition module 702, the second acquisition module 703 and the second input module 704 are shown in FIG. Figure 8The processor 901 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 902, that is, the file conversion method in the above method embodiment is implemented.

[0125] The memory 902 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device of the risk control method, etc. In addition, the memory 902 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 902 may optionally include a memory remotely arranged relative to the processor 901, and these remote memories may be connected to the electronic device of the risk control method via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0126] The electronic device for the intention recognition processing and confidence judgment model generation method may also include: an input device 903 and an output device 904. The processor 901, the memory 902, the input device 903 and the output device 904 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0127] The input device 903 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device of the file conversion method, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 904 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0128] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0130] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0131] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0132] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short).

[0133] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0134] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0135] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for processing intention recognition, characterized in that: Applied to the intelligent outbound call system, the intention recognition processing method includes: Outputting a first inquiry voice, generating a text to be recognized according to an answer voice corresponding to the first inquiry voice, and inputting the text to be recognized into an intent recognition model to obtain an intent recognition result; Obtaining a final result feature and an intermediate result feature of the intent recognition model, wherein the final result feature is generated according to the intent recognition result, the final result feature includes: a highest score corresponding to the intent recognition result, and a difference between the highest score and the second highest score corresponding to the intent recognition result, and the intermediate result feature includes: a vector output by a hidden layer of the intent recognition model; Acquire sentence structure features of the text to be recognized; The final result features, the intermediate result features and the sentence structure features are input into the confidence judgment model to obtain a confidence result corresponding to the intention recognition result. If the confidence result is yes, the intention recognition result is confirmed as the target intention recognition result, and a second inquiry voice is output according to the target intention recognition result. If the confidence result is no, the first inquiry voice is re-output.

2. The intention recognition processing method according to claim 1, characterized in that: The final result feature also includes: an intent recognition vector corresponding to the intent recognition result.

3. The intention recognition processing method according to claim 1, characterized in that: The sentence structure features include at least one of the following: Sentence length, sentence perplexity, and sentence analysis results.

4. The intention recognition processing method according to claim 1, characterized in that: Generating a text to be recognized according to the answer voice corresponding to the first inquiry voice includes: Obtaining the user's answer voice to the first inquiry voice; The text to be recognized is generated according to the answering speech.

5. A method for generating a confidence judgment model, characterized in that: Applied to the intelligent outbound call system, the generation method includes: Outputting a sample inquiry speech, generating a sample text according to a sample answer speech corresponding to the sample inquiry speech, and inputting the sample text into an intent recognition model to obtain a sample intent recognition result; Obtaining a sample final result feature and a sample intermediate result feature of the intent recognition model, wherein the sample final result feature is generated according to the sample intent recognition result, the sample final result feature includes: a highest score corresponding to the sample intent recognition result, and a difference between the highest score and the second highest score corresponding to the sample intent recognition result, and the sample intermediate result feature includes: a vector output by a hidden layer of the intent recognition model; Acquire sample sentence structure features of the sample text; Inputting the sample final result feature, the sample intermediate result feature and the sample sentence structure feature into the confidence judgment model to be trained to obtain a sample confidence result corresponding to the sample intention recognition result, and if the sample confidence result is negative, re-outputting the sample inquiry speech; According to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text, the parameters of the confidence judgment model to be trained are trained to obtain the confidence judgment model.

6. The generation method according to claim 5, characterized in that: The sample final result feature also includes: an intent recognition vector corresponding to the sample intent recognition result.

7. The generation method according to claim 5, characterized in that: The sample sentence structure features include at least one of the following: Sentence length, sentence perplexity, and sentence analysis results.

8. The generation method according to claim 5, characterized in that: Generating the sample text according to the sample answer voice corresponding to the sample inquiry voice includes: Obtaining a sample answer voice of the user in response to the sample inquiry voice; The sample text is generated according to the sample answering speech.

9. An intention recognition processing device, characterized in that: Applied to the intelligent outbound call system, the intention recognition processing device includes: A first input module is configured to output a first inquiry voice, generate a text to be recognized according to an answer voice corresponding to the first inquiry voice, and input the text to be recognized into an intent recognition model to obtain an intent recognition result; A first acquisition module is configured to acquire a final result feature and an intermediate result feature of the intent recognition model, wherein the final result feature is generated according to the intent recognition result, the final result feature includes: a highest score corresponding to the intent recognition result, and a difference between the highest score and the second highest score corresponding to the intent recognition result, and the intermediate result feature includes: a vector output by a hidden layer of the intent recognition model; A second acquisition module is configured to acquire sentence structure features of the text to be recognized; The second input module is configured to input the final result feature, the intermediate result feature and the sentence structure feature into the confidence judgment model to obtain a confidence result corresponding to the intention recognition result. If the confidence result is yes, the intention recognition result is confirmed as a target intention recognition result, and a second inquiry voice is output according to the target intention recognition result. If the confidence result is no, the first inquiry voice is re-output.

10. A device for generating a confidence judgment model, characterized in that: Applied to the intelligent outbound call system, the generating device includes: A third input module is configured to generate a sample text according to a sample answer voice corresponding to the sample inquiry voice, and input the sample text into the intent recognition model to obtain a sample intent recognition result; A third acquisition module is configured to acquire a sample final result feature and a sample intermediate result feature of the intent recognition model, wherein the sample final result feature is generated according to the sample intent recognition result, the sample final result feature includes: a highest score corresponding to the sample intent recognition result, and a difference between the highest score and the second highest score corresponding to the sample intent recognition result, and the sample intermediate result feature includes: a vector output by a hidden layer of the intent recognition model; A fourth acquisition module is configured to acquire sample sentence structure features of the sample text; a fourth input module, configured to input the sample final result feature, the sample intermediate result feature and the sample sentence structure feature into a confidence judgment model to be trained, so as to obtain a sample confidence result corresponding to the sample intent recognition result, and if the sample confidence result is negative, re-output the sample inquiry speech; The training module is configured to train the parameters of the confidence judgment model to be trained according to the sample confidence result, the sample intent recognition result and the actual intent recognition result corresponding to the sample text to obtain the confidence judgment model.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

12. A computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 8.

Images