Intention recognition model training method and apparatus, computer device, and storage medium

Abstract

The application discloses an intention recognition model training method and device, computer equipment and a storage medium, the method comprising: inputting sample text into an intention recognition model to obtain N intention probability values; determining K target intention categories and N-K non-target intention categories in N preset intention categories according to an actual intention category; calculating a loss function according to the K target intention categories, the N-K non-target intention categories and a preset probability threshold, adjusting parameters of the intention recognition model according to the loss function until the loss function converges; calculating the accuracy of the preset probability threshold and determining whether the preset condition is met, and if the accuracy meets the preset condition, outputting the intention recognition model after parameter updating. The network model can improve the speed and efficiency of intention recognition, reduce the labor cost and time cost, and the accuracy of the intention recognition model can be improved by adjusting the parameters of the intention recognition model and determining whether the accuracy of the preset probability threshold meets the preset condition.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an intention recognition model training method, apparatus, computer equipment, and storage medium. Background Technology

[0002] Communication with customers always plays a crucial role in business operations. During these interactions, timely and accurate identification of customer intent based on their responses is paramount. Large enterprises, with their highly complex business systems encompassing numerous scenarios and multiple potential intents within each scenario, and each customer's response containing various intentions, often struggle to accurately identify the diverse intents within a large pool of responses. This results in low accuracy for existing multi-objective intent recognition models, hindering sales staff from providing timely and appropriate service to customers. Summary of the Invention

[0003] This invention provides an intent recognition model training method, apparatus, computer device, and storage medium to solve the problem of low recognition accuracy of multi-target intent recognition models.

[0004] A method for training an intent recognition model, comprising:

[0005] Obtain sample text of the intent to be identified, input the sample text into the intent recognition model, and obtain N intent probability values ​​of the sample text belonging to N preset intent categories. The sample text includes the customer's reply text, and the preset intent categories include N of the following: policy cancellation, policy renewal, accident, progress, product consultation, and process consultation, where N is an integer greater than 2.

[0006] Obtain the actual intent category of the sample text, and determine K target intent categories and NK non-target intent categories from the N preset intent categories based on the actual intent category. The actual intent category includes K of the preset intent categories, where K is an integer greater than 1 and K≤N.

[0007] A loss function is calculated based on K target intent categories, NK non-target intent categories, and a preset probability threshold. The parameters of the intent recognition model are then adjusted based on the loss function until the loss function converges.

[0008] A preset intent category corresponding to an intent probability value greater than the preset probability threshold is determined as the predicted target intent category, and the accuracy of the preset probability threshold is calculated based on the predicted target intent category and the actual intent category.

[0009] Determine whether the accuracy rate meets the preset conditions. If the accuracy rate meets the preset conditions, output the updated intent recognition model.

[0010] An intent recognition model training device, comprising:

[0011] A probability value determination model is used to obtain sample text of the intent to be identified. The sample text is input into the intent recognition model to obtain N intent probability values ​​of the sample text belonging to N preset intent categories. The sample text includes the customer's reply text. The preset intent categories include N of the following: policy cancellation, policy renewal, accident, progress, product consultation, and process consultation, where N is an integer greater than 2.

[0012] The target intent category determination module is used to obtain the actual intent category of the sample text, and determine K target intent categories and NK non-target intent categories from the N preset intent categories based on the actual intent category. The actual intent category includes K of the preset intent categories, where K is an integer greater than 1 and K≤N.

[0013] The parameter adjustment module is used to calculate a loss function based on K target intent categories, NK non-target intent categories and a preset probability threshold, and adjust the parameters of the intent recognition model according to the loss function until the loss function converges.

[0014] The sample text information calculation module is used to determine the preset intent category corresponding to the intent probability value that is greater than the preset probability threshold as the predicted target intent category, and to calculate the accuracy of the preset probability threshold based on the predicted target intent category and the actual intent category.

[0015] The intent recognition model determination module is used to determine whether the accuracy rate meets the preset conditions. If the accuracy rate meets the preset conditions, it outputs the intent recognition model with updated parameters.

[0016] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described intention recognition model training method.

[0017] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described intention recognition model training method.

[0018] The aforementioned intent recognition model training method, apparatus, computer equipment, and storage medium input sample text into the intent recognition model to obtain N intent probability values ​​for the sample text belonging to N preset intent categories. This improves the speed and efficiency of intent recognition through a network model, reducing the human and time costs associated with intent recognition. Based on the actual intent category, K target intent categories and NK non-target intent categories are determined from the N preset intent categories. A loss function is calculated based on the K target intent categories, NK non-target intent categories, and a preset probability threshold. The parameters of the intent recognition model are adjusted according to the loss function until convergence. The parameters of the intent recognition model are further adjusted based on the actual intent category to improve the accuracy of the intent recognition model. The accuracy of the preset probability threshold is calculated. If the accuracy meets the preset conditions, the updated intent recognition model is output. The accuracy of the intent recognition model is further guaranteed by checking whether the accuracy of the preset probability threshold meets the preset conditions. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of an application environment for an intent recognition model training method according to an embodiment of the present invention;

[0021] Figure 2 This is a flowchart of an intent recognition model training method according to an embodiment of the present invention;

[0022] Figure 3 This is another flowchart of an intent recognition model training method in one embodiment of the present invention;

[0023] Figure 4 This is another flowchart of an intent recognition model training method in one embodiment of the present invention;

[0024] Figure 5 This is another flowchart of an intent recognition model training method in one embodiment of the present invention;

[0025] Figure 6 This is another flowchart of an intent recognition model training method in one embodiment of the present invention;

[0026] Figure 7 This is another flowchart of an intent recognition model training method in one embodiment of the present invention;

[0027] Figure 8This is another flowchart of an intent recognition model training method in one embodiment of the present invention;

[0028] Figure 9 This is a schematic diagram of an intent recognition model training device according to an embodiment of the present invention;

[0029] Figure 10 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] The intent recognition model training method provided in this embodiment of the invention can be applied to, for example... Figure 1 The application environment is shown. Specifically, this intent recognition model training method is applied in an intent recognition processing system, which includes, as shown in the example, [example of such system]. Figure 1 The client and server shown communicate over a network. Based on standard intent recognition codes, they can quickly identify the intent of customer responses, improving the accuracy of intent recognition and ensuring that sales staff can respond to and serve customers promptly and accurately, meeting business needs. The client, also known as the user terminal, is the program that provides local services to customers, corresponding to the server. The client can be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0032] In one embodiment, such as Figure 2 As shown, an intent recognition model training method is provided, which is then applied to... Figure 1 Taking the server in the example, the following steps are included:

[0033] S201: Obtain the sample text of the intent to be identified, input the sample text into the intent recognition model, and obtain N intent probability values ​​of the sample text belonging to N preset intent categories. The sample text includes the customer's reply text, and the preset intent categories include N of the following: policy cancellation, policy renewal, accident, progress, product consultation, and process consultation, where N is an integer greater than 2.

[0034] S202: Obtain the actual intent category of the sample text, and determine K target intent categories and NK non-target intent categories from N preset intent categories based on the actual intent category. The actual intent category includes K of the preset intent categories, where K is an integer greater than 1 and K≤N;

[0035] S203: Calculate the loss function based on K target intent categories, NK non-target intent categories and a preset probability threshold, and adjust the parameters of the intent recognition model according to the loss function until the loss function converges;

[0036] S204: Determine the preset intent category corresponding to the intent probability value that is greater than the preset probability threshold as the predicted target intent category, and calculate the accuracy of the preset probability threshold based on the predicted target intent category and the actual intent category.

[0037] S205: Determine whether the accuracy meets the preset conditions. If the accuracy meets the preset conditions, output the updated intent recognition model.

[0038] The sample text for identifying intent includes the customer's response text, which can be obtained from various communication methods between the salesperson and the customer. For example, in telephone communication, the customer's voice response can be obtained and converted into corresponding text using speech recognition technology; in text communication, the customer's response text can be obtained directly. Intent categories represent the business-related intents that customers may exhibit during communication with the salesperson. These intent categories vary depending on the business area. For example, in the insurance business, intent categories include policy cancellation, renewal, claims, progress, product inquiries, and process inquiries.

[0039] As an example, in step S201, after obtaining sample text of the intent to be identified during the communication between the salesperson and the customer, the sample text is input into the intent recognition model to obtain N intent probability values ​​for the sample text belonging to N preset intent categories. Here, N is an integer greater than 1, and the N preset intent categories are N intent categories from policy cancellation, policy renewal, claim, progress, product consultation, and process consultation. The specific intent categories selected can be determined according to the actual situation. The intent recognition model outputs N intent probability values ​​to represent the probability that the customer possesses the corresponding N preset intent categories during the communication process.

[0040] As an example, in step S202, for the sample text in the training of the intent recognition model, the actual intent category of the sample text is first obtained, and the number of actual intent categories of the sample text is recorded as K. In the insurance business field, the actual intent is the real intent expressed by the customer in the sample text, which can be judged and determined manually by the salesperson. The actual intent category includes K intent categories from N preset intent categories. For N preset intent categories, the K preset intent categories that belong to the corresponding actual intent category are recorded as K target intent categories, and the NK preset intent categories that do not belong to the corresponding actual intent category are recorded as NK non-target intent categories. Thus, the K target intent categories and NK non-target intent categories from N preset intent categories are determined according to the actual intent category of the sample text.

[0041] Where K is an integer greater than 1, K≤N. The number and categories of actual intent categories corresponding to different sample texts are not the same. Therefore, the numerical value K of different sample texts is not the same, and the K target intent categories and NK non-target intent categories are also not the same.

[0042] As an example, in step S203, in order to improve the recognition accuracy of the intent recognition model, when calculating the loss function based on K target intent categories, NK non-target intent categories and a preset probability threshold, specifically, the differences between the K intent probability values ​​corresponding to the K target intent categories, the NK intent probability values ​​corresponding to the NK non-target intent categories and the preset probability threshold can be calculated. It is desirable to ensure that the intent probability value corresponding to each target intent category is greater than the intent probability value corresponding to each non-target intent category, so as to guarantee the recognition accuracy of the intent recognition model. The parameters of the intent recognition model are adjusted according to the loss function until the loss function converges, so as to gradually improve the recognition accuracy of the intent recognition model.

[0043] The preset probability threshold is set in advance based on the actual situation. For the N intent probability values ​​output by the intent recognition model, when determining the predicted target intent category of the sample text, the preset intent category corresponding to the intent probability value greater than the preset probability threshold can be taken as the predicted target intent category of the sample text, and the preset intent category corresponding to the intent probability value not greater than the preset probability threshold can be taken as the predicted non-target intent category of the sample text. Therefore, in the training process of the intent recognition model, the accuracy of the N intent probability values ​​is characterized by calculating the difference between the K intent probability values ​​corresponding to the K target intent categories and the preset probability threshold, combined with the difference between the preset probability threshold and the NK intent probability values ​​corresponding to the NK non-target intent categories. The parameters of the intent recognition model are adjusted according to the loss function until the loss function converges. By making the intent probability value corresponding to each target intent category greater than the intent probability value corresponding to each non-target intent category, the recognition accuracy of the intent recognition model is gradually improved.

[0044] As an example, in step S204, for the N intent probability values ​​output by the intent recognition model, when an intent probability value is greater than a preset probability threshold, the preset intent category corresponding to that intent probability value can be regarded as the predicted target intent category of the sample text. In this example, the accuracy of the preset probability threshold can be calculated based on the predicted target intent category and the actual intent category. Furthermore, given a determined actual intent category, the more instances where the predicted target intent category matches the actual intent category, the higher the accuracy of the predicted target intent category determined by the corresponding preset probability threshold. This indicates a higher recognition accuracy for the intent recognition model using the preset probability threshold. Therefore, calculating the accuracy corresponding to the preset probability threshold by predicting the target intent category and the actual intent category improves the recognition accuracy of the intent recognition model.

[0045] As an example, in step S205, it is first determined whether the accuracy meets the preset conditions. If the accuracy meets the preset conditions, it indicates that the recognition accuracy of the intent recognition model using the corresponding preset probability threshold is high, and there is no need to further optimize the intent recognition model by adjusting the value of the preset probability threshold. The updated intent recognition model can then be output as a trained intent recognition model, used to recognize the text to be recognized, enabling sales personnel to respond to the customer's intent promptly and accurately, thus improving business efficiency. In this example, the preset conditions can be set according to the actual situation. For example, the preset condition is an accuracy threshold. When the accuracy is not less than the accuracy threshold, it is determined that the accuracy meets the preset conditions, and the trained intent recognition model can be obtained.

[0046] In the intent recognition model training method provided in this embodiment, sample text is input into the intent recognition model to obtain N intent probability values ​​for the sample text belonging to N preset intent categories. The network model improves the speed and efficiency of intent recognition, reducing the human and time costs of intent recognition. Based on the actual intent category, K target intent categories and NK non-target intent categories are determined from the N preset intent categories. A loss function is calculated based on the K target intent categories, NK non-target intent categories, and a preset probability threshold. The parameters of the intent recognition model are adjusted according to the loss function until the loss function converges. The parameters of the intent recognition model are further adjusted based on the actual intent category to improve the accuracy of the intent recognition model. The accuracy of the preset probability threshold is calculated. It is determined whether the accuracy meets a preset condition. If the accuracy meets the preset condition, the updated intent recognition model is output. The accuracy of the intent recognition model is further guaranteed by determining whether the accuracy of the preset probability threshold meets the preset condition.

[0047] In one embodiment, such as Figure 3 As shown, the intent recognition model includes an encoder and a fully connected layer, using sample text of the intent to be recognized as training samples and the actual intent category as training labels.

[0048] The training process for the intent recognition model includes:

[0049] S301: Input the sample text into the encoder for feature extraction to obtain the sample text features;

[0050] S302: Input the sample text features into the fully connected layer for feature mapping to obtain N intent probability values ​​for the sample text belonging to N preset intent categories.

[0051] In this example, the training samples are a large number of sample texts containing the intent to be identified, and the training labels are the K actual intent categories of the sample texts. The actual intent categories are used to evaluate the N intent probability values ​​obtained, so that the intent recognition model can be trained based on the evaluation results to obtain a trained intent recognition model.

[0052] As an example, in step S301, during the training process of the intent recognition model, sample text is input into the encoder for feature extraction to obtain sample text features, which are used to characterize the intent features of the sample text.

[0053] As an example, in step S302, the sample text features are input into a fully connected layer for feature mapping. The feature mapping result is normalized using a normalized exponential function. The normalization result is the N probability values ​​of the sample text belonging to N preset intent categories.

[0054] In the intent recognition model training method provided in this embodiment, the intent recognition model includes an encoder and a decoder. The sample text of the intent to be recognized is used as the training sample, and the actual intent category is used as the training label to train the intent recognition model, which can effectively improve the recognition accuracy of the intent recognition model.

[0055] In one embodiment, such as Figure 4 As shown, step S203, which calculates the loss function based on K target intent categories, NK non-target intent categories, and a preset probability threshold, includes:

[0056] S401: Initialize the number of iterations to 0, perform the first round of model training using the preset probability threshold, and obtain the updated probability threshold after the first round of model training is completed;

[0057] S402: Increment the iteration count by 1, use the updated probability threshold to train the model for the second round, and obtain the updated probability threshold after the second round of model training is completed;

[0058] S403: Continue in this manner until the probability threshold of a certain update meets the preset conditions, and output the trained intent recognition model.

[0059] In the training process of the intent recognition model, a loss function calculated based on K target intent categories, NK non-target intent categories, and a preset probability threshold can be used to adjust the parameters of the intent recognition model. The preset probability threshold is used to distinguish between target intent categories and non-target intent categories, and can be used in the calculation of the loss function to measure the difference between the consciousness probability values ​​corresponding to the target intent categories and non-target intent categories. In the training process of the intent recognition model, the preset probability threshold needs to be trained and optimized to improve the recognition accuracy of the intent recognition model and obtain a well-trained intent recognition model.

[0060] As an example, in step S401, during the training process of the intent recognition model, the number of iterations is first initialized to 0, the first round of model training is performed using a preset probability threshold, and an updated probability threshold is obtained after the first round of model training is completed.

[0061] The preset probability threshold can be set according to the actual situation. In this example, the preset probability threshold is 0.5.

[0062] As an example, in step S402, after completing the first round of model training for the intent recognition model, an updated probability threshold is obtained, the iteration count is incremented by 1, and the iteration count changes from 0 to 1. The updated probability threshold is used for the second round of model training, and an updated probability threshold is obtained after the second round of model training is completed.

[0063] As an example, in step S403, after completing the second round of model training for the intent recognition model, an updated probability threshold is obtained again. This process is repeated, using the updated probability threshold to perform multiple rounds of model training, continuously obtaining updated probability thresholds until a certain updated probability threshold meets a preset condition. At this point, the training of the intent recognition model can be terminated, and the trained intent recognition model can be output.

[0064] Training of the intent recognition model can be terminated when the accuracy corresponding to the updated probability threshold meets the preset conditions.

[0065] In the intent recognition model training method provided in this embodiment, when the number of iterations is initialized to 0, the first round of model training is performed using a preset probability threshold to obtain an updated probability threshold. The number of iterations is then incremented by 1, and the second round of model training is performed using the updated probability threshold to obtain another updated probability threshold. This process continues until the updated probability threshold meets a preset condition, at which point the trained intent recognition model is output. By adjusting and optimizing the preset probability threshold, the accuracy of classifying target intent categories and non-target intent categories can be improved, thereby continuously improving the recognition accuracy of the intent recognition model.

[0066] In one embodiment, such as Figure 5 As shown, step S203, which calculates the loss function based on K target intent categories, NK non-target intent categories, and a preset probability threshold, includes:

[0067] S501: Take the K intent probability values ​​corresponding to the K target intent categories as the target intent probability values, and take the NK intent probability values ​​corresponding to the NK non-target intent categories as the non-target intent probability values;

[0068] S502: Calculate the loss function based on K target intent probability values, NK non-target intent probability values, and a preset probability threshold. The loss function Loss is:

[0069]

[0070] In the formula, p0 is the preset probability threshold, K is the number of target intent probability values, NK is the number of non-target intent probability values, and p i Let p be the probability value of the i-th target intent. j Let be the probability value of the j-th non-target intent.

[0071] The loss function, calculated based on K target intent categories, NK non-target intent categories, and a preset probability threshold, aims to ensure that the intent probability value corresponding to each target intent category is greater than the intent probability value corresponding to each non-target intent category, thereby guaranteeing the recognition accuracy of the intent recognition model.

[0072] As an example, in step S501, the K intent probability values ​​corresponding to the K target intent categories are used as target intent probability values, and the NK intent probability values ​​corresponding to the NK non-target intent categories are used as non-target intent probability values.

[0073] As an example, in step S502, since it is desired that the probability value of the intent corresponding to each target intent category is greater than the probability value of the intent corresponding to each non-target intent category, a preset probability threshold is used as an intermediate benchmark. Based on the degree to which the probability values ​​of K target intents are greater than the preset probability threshold, combined with the degree to which the preset probability threshold is greater than the probability values ​​of NK non-target intents, the loss function of the intent recognition model is obtained, denoted as Loss. The loss function Loss is:

[0074]

[0075] In the formula, p0 is the preset probability threshold, K is the number of target intent probability values, NK is the number of non-target intent probability values, and p i Let p be the probability value of the i-th target intent. j Let be the probability value of the j-th non-target intent.

[0076] The parameters of the intent recognition model can then be adjusted according to the loss function until the loss function converges, thus completing the adjustment of the model parameters and improving the recognition accuracy of the intent recognition model.

[0077] In the intent recognition model training method provided in this embodiment, K target intent probability values ​​and NK non-target intent probability values ​​are first determined, and a loss function is calculated based on the K target intent probability values, NK non-target intent probability values ​​and a preset probability threshold. The parameters of the intent recognition model are adjusted based on the loss function, which greatly improves the recognition accuracy of the intent recognition model.

[0078] In one embodiment, such as Figure 6 As shown, step S204, which calculates the accuracy of a preset probability threshold based on the target intent category and the actual intent category, includes:

[0079] S601: Determine the number of correct categories in the predicted target intent category based on the actual intent category;

[0080] S602: Calculate the ratio of the number of correct categories to the total number of target intent categories, and use the ratio as the accuracy.

[0081] As an example, in step S601, the actual intent category and the predicted target intent category are first compared to determine the number of correct categories in the predicted target intent category. The more correct categories there are, the higher the accuracy of the preset probability threshold.

[0082] As an example, in step S602, the ratio of the number of correct categories to the total number of actual intent categories is calculated, and the ratio is used as the accuracy rate. This accuracy rate is used to characterize the content of correct categories in the target intent categories determined according to a preset probability threshold. The higher the accuracy rate, the higher the recognition accuracy of the intent recognition model using the preset probability threshold.

[0083] As an example, in step S602, the ratio of the number of correct categories to the total number of preset intent categories is calculated, and the ratio is used as the accuracy rate. This accuracy rate is used to characterize the content of correct categories in the target intent categories determined according to the preset probability threshold. The higher the accuracy rate, the higher the recognition accuracy of the intent recognition model using the preset probability threshold.

[0084] In the intent recognition model training method provided in this embodiment, the number of correct categories in the predicted target intent category is determined according to the actual intent category; the ratio of the number of correct categories to the total number of target intent categories is calculated, and the ratio is used as the accuracy. The accuracy represents the recognition accuracy of the intent recognition model using the corresponding preset probability threshold, thereby improving the training flexibility and diversity of the intent recognition model.

[0085] In one embodiment, such as Figure 7 As shown, step S602, which determines whether the accuracy meets the preset conditions, includes:

[0086] S701: If the accuracy rate is not less than the preset accuracy rate threshold, then the accuracy rate is determined to meet the preset condition.

[0087] S702: If the accuracy is less than the preset accuracy threshold, then the accuracy is determined not to meet the preset conditions.

[0088] The accuracy threshold can be preset according to the actual situation. In this embodiment, the accuracy threshold is preset to 0.8 according to the actual situation.

[0089] As an example, in step S701, if the accuracy rate is not less than the preset accuracy rate threshold, it indicates that the correct category content in the target intent category determined according to the preset probability threshold is high, which means that the recognition accuracy of the intent recognition model using the preset probability threshold is higher, and the determination accuracy meets the preset condition.

[0090] As an example, in step S702, if the accuracy is less than the preset accuracy threshold, it indicates that the correct category content in the target intent category determined according to the preset probability threshold is low, which means that the lower the recognition accuracy of the intent recognition model using the preset probability threshold, the more accurate the judgment does not meet the preset condition.

[0091] In the intent recognition model training method provided in this embodiment, if the accuracy is not less than a preset accuracy threshold, the accuracy is determined to meet the preset condition; if the accuracy is less than the preset accuracy threshold, the accuracy is determined to not meet the preset condition. The recognition accuracy of the intent recognition model is indicated by the content of the correct category in the target intent category, which greatly improves the training flexibility and diversity of the intent recognition model.

[0092] In one embodiment, such as Figure 8 As shown, in step S702, if the accuracy is less than a preset accuracy threshold, it is determined that the accuracy does not meet the preset condition, including:

[0093] S801: If the accuracy does not meet the preset conditions, a random walk strategy is used to adjust the preset probability threshold.

[0094] S802: Recalculate the accuracy based on the adjusted preset probability threshold until the accuracy meets the preset conditions, and determine the intent recognition model corresponding to the adjusted preset probability threshold as the trained intent recognition model.

[0095] The preset probability threshold is used to determine the predicted target intent category from N preset intent categories based on N intent probability values. When the preset probability threshold changes, the predicted target intent category changes accordingly. Therefore, when the accuracy does not meet the preset conditions, the predicted target intent category can be changed by adjusting the value of the preset probability threshold, thereby improving the accuracy corresponding to the preset probability threshold.

[0096] As an example, in step S801, if the accuracy does not meet the preset conditions, a random walk strategy is used to adjust the preset probability threshold in the intent recognition model. The random walk strategy can refer to randomly sampling the increase and decrease of the preset probability threshold, with the step size of the increase and decrease being a preset value, such as 0.1. In this example, the probability of increasing the preset probability threshold and the probability of decreasing the preset probability threshold are both set to 0.5.

[0097] As an example, in step S802, the predicted target intent category is re-determined based on the adjusted preset probability threshold, and the accuracy is recalculated until the accuracy meets the preset conditions. This determines that the intent recognition model corresponding to the adjusted preset probability threshold is the trained intent recognition model, thereby significantly improving the recognition accuracy of the intent recognition model.

[0098] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0099] In one embodiment, an intent recognition model training device is provided, which corresponds one-to-one with the intent recognition model training method described in the above embodiments. For example... Figure 9 As shown, the intent recognition model training device includes a probability value determination module 901, a target intent category determination module 902, a parameter adjustment module 903, a sample text information calculation module 904, and an intent recognition model determination module 905. Detailed descriptions of each functional module are as follows:

[0100] The probability value determination model 901 is used to obtain sample text of the intent to be identified. The sample text is input into the intent recognition model to obtain N intent probability values ​​of the sample text belonging to N preset intent categories. The sample text includes the customer's reply text. The preset intent categories include N of the following: policy cancellation, policy renewal, accident, progress, product consultation, and process consultation, where N is an integer greater than 2.

[0101] The target intent category determination module 902 is used to obtain the actual intent category of the sample text, and determine K target intent categories and NK non-target intent categories from N preset intent categories based on the actual intent category. The actual intent categories include K of the preset intent categories, where K is an integer greater than 1 and K≤N.

[0102] The parameter adjustment module 903 is used to calculate the loss function based on K target intent categories, NK non-target intent categories and a preset probability threshold, and adjust the parameters of the intent recognition model according to the loss function until the loss function converges.

[0103] The sample text information calculation module 904 is used to determine the preset intent category corresponding to the intent probability value that is greater than the preset probability threshold as the predicted target intent category, and to calculate the accuracy of the preset probability threshold based on the predicted target intent category and the actual intent category.

[0104] The intent recognition model determination module 905 is used to determine whether the accuracy meets the preset conditions. If the accuracy meets the preset conditions, it outputs the intent recognition model with updated parameters.

[0105] In one embodiment, the probability value determination model 901 includes:

[0106] The sample text feature extraction submodule is used to input sample text into the encoder for feature extraction to obtain sample text features;

[0107] The intent probability value determination submodule is used to input the sample text features into the fully connected layer for feature mapping, and obtain N intent probability values ​​for the sample text to belong to N preset intent categories.

[0108] In one embodiment, the parameter adjustment module 903 includes:

[0109] The first model training submodule is used to initialize the number of iterations to 0, perform the first round of model training using a preset probability threshold, and obtain the updated probability threshold after the first round of model training is completed.

[0110] The second model training submodule is used to increment the iteration count by 1, use the updated probability threshold to perform the second round of model training, and obtain the updated probability threshold after the second round of model training is completed.

[0111] The third model training submodule is used to repeat this process until the probability threshold of a certain update meets the preset conditions, and then output the trained intent recognition model.

[0112] In one embodiment, the parameter adjustment module 903 includes:

[0113] The intent probability value partitioning submodule is used to take the K intent probability values ​​corresponding to the K target intent categories as target intent probability values ​​and the NK intent probability values ​​corresponding to the NK non-target intent categories as non-target intent probability values.

[0114] The loss function determination submodule is used to calculate the loss function based on K target intent probability values, NK non-target intent probability values, and a probability threshold. The loss function Loss is:

[0115]

[0116] In the formula, p0 is the probability threshold, K is the number of target intent probability values, NK is the number of non-target intent probability values, and p i Let p be the probability value of the i-th target intent. j Let be the probability value of the j-th non-target intent.

[0117] In one embodiment, the sample text information calculation module 904 includes:

[0118] The Correct Category Count Determination submodule is used to determine the number of correct categories in the predicted target intent category based on the actual intent category.

[0119] The accuracy calculation submodule is used to calculate the ratio of the number of correct categories to the total number of target intent categories, and the ratio is used as the accuracy.

[0120] In one embodiment, the accuracy calculation submodule includes:

[0121] The first accuracy comparison unit is used to determine that the accuracy meets the preset condition if the accuracy is not less than the preset accuracy threshold.

[0122] The second accuracy comparison unit is used to determine that the accuracy does not meet the preset conditions if the accuracy is less than the preset accuracy threshold.

[0123] In one embodiment, the second accuracy comparison unit includes:

[0124] The first probability threshold adjustment subunit is used to adjust the preset probability threshold by using a random walk strategy if the accuracy does not meet the preset conditions.

[0125] The second probability threshold adjustment subunit is used to recalculate the accuracy based on the adjusted preset probability threshold until the accuracy meets the preset conditions, and to determine the intent recognition model corresponding to the adjusted preset probability threshold as the trained intent recognition model.

[0126] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database is used to generate a database during the training of an intent recognition model. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an intent recognition model training method.

[0127] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the intent recognition model training method described in the above embodiments, for example... Figure 2 As shown in S201-S205, or Figures 3 to 8 As shown, to avoid repetition, it will not be described again here. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in this embodiment of the intent recognition model training device, for example... Figure 9 The functions of the probability value determination model 901, target intent category determination module 902, parameter adjustment module 903, sample text information calculation module 904, and intent recognition model determination module 905 shown are not described in detail here to avoid repetition.

[0128] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the intent recognition model training method described in the above embodiments, for example... Figure 2 As shown in S201-S205, or Figures 3 to 8As shown, to avoid repetition, it will not be described again here. Alternatively, when the computer program is executed by the processor, it implements the functions of each module / unit in this embodiment of the intent recognition model training method, for example... Figure 9 The functions of the probability value determination model 901, target intent category determination module 902, parameter adjustment module 903, sample text information calculation module 904, and intent recognition model determination module 905 shown are not described again here to avoid repetition. The computer-readable storage medium can be non-volatile or volatile.

[0129] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0130] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0131] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for training an intent recognition model, the method comprising: The method comprises the following steps: obtaining a sample text of an to-be-identified intent, inputting the sample text into an intent recognition model, and obtaining N intent probability values of the sample text belonging to N preset intent categories, wherein the sample text comprises a reply text of a customer, and the preset intent categories comprise N of the following: refund, renewal, claim, progress, product consultation, and process consultation, and N is an integer greater than 2; obtaining an actual intent category of the sample text, determining K target intent categories and N-K non-target intent categories in the N preset intent categories according to the actual intent category, wherein the actual intent category comprises K of the preset intent categories, and K is an integer greater than 1, and K≤N; calculating a loss function according to the K target intent categories, the N-K non-target intent categories, and a preset probability threshold, adjusting parameters of the intent recognition model according to the loss function, and stopping until the loss function converges; determining that a preset intent category corresponding to an intent probability value greater than the preset probability threshold is a predicted target intent category, and calculating an accuracy rate of the preset probability threshold according to the predicted target intent category and the actual intent category; determining whether the accuracy rate meets a preset condition, and outputting the intent recognition model after parameter updating if the accuracy rate meets the preset condition; the method comprises the following steps: taking K intent probability values corresponding to the K target intent categories as target intent probability values, and taking N-K intent probability values corresponding to the N-K non-target intent categories as non-target intent probability values; calculating a loss function according to the K target intent probability values, the N-K non-target intent probability values, and the preset probability threshold, wherein the loss function Loss is: In the formula, is the preset probability threshold, K is the number of the target intent probability values, N-K is the number of the non-target intent probability values, is the i-th target intent probability value, is the j-th non-target intent probability value. 2.The intention recognition model training method of claim 1, wherein, the intent recognition model comprises an encoder and a fully connected layer, a sample text of an to-be-identified intent is used as a training sample, and an actual intent category is used as a training label; the training process of the intent recognition model comprises the following steps: inputting the sample text into the encoder to extract features, and obtaining sample text features; inputting the sample text features into the fully connected layer to map features, and obtaining N intent probability values of the sample text belonging to N preset intent categories. 3.The intention recognition model training method of claim 1, wherein, the method comprises the following steps: initializing an iteration number to 0, performing first-round model training by using the preset probability threshold, obtaining an updated probability threshold after the first-round model training is completed; increasing the iteration number by 1, performing second-round model training by using the updated probability threshold, obtaining a further updated probability threshold after the second-round model training is completed; and so on, until a preset condition is met for a certain updated probability threshold, and outputting a trained intent recognition model. 4.The intention recognition model training method of claim 1, wherein, the method comprises the following steps: determining the number of correct categories in the predicted target intent category according to the actual intent category; Calculate a ratio of the correct class quantity and a total quantity of the actual intent classes, and take the ratio as the accuracy. 5.The intention recognition model training method of claim 1, wherein, The judging whether the accuracy meets a preset condition comprises: If the accuracy is not less than a preset accuracy threshold, it is determined that the accuracy meets the preset condition; If the accuracy is less than the preset accuracy threshold, it is determined that the accuracy does not meet the preset condition. 6.The intention recognition model training method of claim 5, wherein, After it is determined that the accuracy does not meet the preset condition if the accuracy is less than the preset accuracy threshold, the method comprises: If the accuracy does not meet the preset condition, a random walk strategy is used to adjust the preset probability threshold; The accuracy is recalculated according to the adjusted preset probability threshold until the accuracy meets the preset condition, and it is determined that an intent recognition model corresponding to the adjusted preset probability threshold is a trained intent recognition model. 7.An intention recognition model training apparatus, comprising: The method comprises: A probability value determination model is configured to obtain a sample text of an intent to be recognized, input the sample text into an intent recognition model, and obtain N intent probability values of the sample text belonging to N preset intent classes, wherein the sample text comprises a reply text of a customer, the preset intent classes comprise N of the following: refund, renewal, claim, progress, product consultation, and process consultation, and N is an integer greater than 2; A target intent class determination module is configured to obtain an actual intent class of the sample text, and determine K target intent classes and N-K non-target intent classes from the N preset intent classes according to the actual intent class, wherein the actual intent class comprises K of the preset intent classes, and K is an integer greater than 1 and K≤N; A parameter adjustment module is configured to calculate a loss function according to the K target intent classes, the N-K non-target intent classes, and a preset probability threshold, and adjust parameters of the intent recognition model according to the loss function until the loss function converges; A sample text information calculation module is configured to determine that a preset intent class corresponding to an intent probability value greater than the preset probability threshold is a predicted target intent class, and calculate an accuracy of the preset probability threshold according to the predicted target intent class and the actual intent class; An intent recognition model determination module is configured to judge whether the accuracy meets a preset condition, and output an intent recognition model with updated parameters if the accuracy meets the preset condition. The parameter adjustment module comprises: An intent probability value division sub-module is configured to take K intent probability values corresponding to the K target intent classes as target intent probability values, and take N-K intent probability values corresponding to the N-K non-target intent classes as non-target intent probability values; A loss function determination sub-module is configured to calculate a loss function according to the K target intent probability values, the N-K non-target intent probability values, and the preset probability threshold, and the loss function Loss is: In the formula, is the preset probability threshold, K is the number of the target intent probability values, N-K is the number of the non-target intent probability values, is the i-th target intent probability value, is the j-th non-target intent probability value.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the intent recognition model training method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program implements the intent recognition model training method according to any one of claims 1 to 6 when executed by the processor.

Images