Intention recognition method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses an intention recognition method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining training data and clustering data, carrying out the comparative learning training of a pre-training language model according to the training data, determining a sentence vector model, inputting the clustering data into the sentence vector model, and obtaining the sentence vector model. And determining corresponding sentence vectors according to the initial parameter combinations, respectively inputting the sentence vectors into the clustering models corresponding to the initial parameter combinations for clustering so as to determine an optimal parameter combination according to each clustering result, and determining an intention category according to the clustering data and the clustering result of the clustering model under the optimal parameter combination. Therefore, the sentence vector model is determined through contrast learning training, the optimal parameter combination of clustering and the corresponding intention category are determined according to the clustering results corresponding to different initial parameter combinations, and the accuracy of new intention recognition can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an intent recognition method, device, electronic device and computer-readable storage medium. Background Art

[0002] Corresponding dialogue task systems are provided in different business scenarios. For example, in the intelligent customer service system in the taxi-hailing scenario, users input questions generated when using the service into the intelligent customer service system for consultation. The system identifies the user's questions and determines the user's intention and gives a corresponding reply based on the user's intention. Therefore, before using the intelligent customer service system, it is necessary to classify the questions raised by users by category and provide them as intention data to the system for model training to improve the system's ability to identify different user intentions and bring users a better user experience.

[0003] Existing new intent recognition methods mainly use vectorization tools to vectorize user questions, and then use traditional machine learning clustering algorithms to cluster user questions. However, when using vectorization tools to vectorize user questions, there may be situations where user questions with similar intents are far apart in the vector space or user questions with dissimilar intents are close to each other in the vector space, which will cause errors in subsequent clustering, or when the model itself has limitations, it will also cause unsatisfactory intent recognition results. For example, taking the traditional machine learning model K-Means clustering as an example, this model is suitable for data where each cluster is distributed in clusters, but the question vectors obtained by the vectorization tool often do not have the characteristics of clusters being distributed in clusters. Summary of the invention

[0004] In view of this, embodiments of the present invention aim to provide an intent recognition method, apparatus, electronic device, and computer-readable storage medium to improve the accuracy of new intent recognition.

[0005] In a first aspect, an embodiment of the present invention is to provide a method for identifying intent, the method comprising:

[0006] Acquire training data and clustering data, wherein the training data includes user sentences and dialogue sentences, and the clustering data includes user sentences;

[0007] Perform comparative learning training on the pre-trained language model according to the training data to determine a sentence vector model;

[0008] Inputting the clustering data into the sentence vector model to determine the corresponding sentence vector;

[0009] Inputting the sentence vectors into the clustering models corresponding to the initial parameter combinations for clustering, so as to determine the best parameter combination according to the clustering results;

[0010] The intent category is determined according to the clustering data and the clustering result of the clustering model under the optimal parameter combination.

[0011] Furthermore, each initial parameter combination is determined based on the following steps:

[0012] The parameter space corresponding to each clustering parameter is traversed in turn to determine each initial parameter combination, wherein the parameter space includes at least one optional value corresponding to the clustering parameter.

[0013] Furthermore, the clustering results include outlier sample points, valid sample points and the number of categories, and determining the best parameter combination according to each of the clustering results includes:

[0014] Screening the initial parameter combinations based on preset screening rules and clustering results corresponding to the initial parameter combinations to determine at least one candidate parameter combination;

[0015] The candidate parameter combination corresponding to the clustering result with the largest number of categories is determined as the optimal parameter combination.

[0016] Furthermore, the screening rules are used to screen out initial parameter combinations corresponding to clustering results whose number of categories is less than a predetermined number, initial parameter combinations corresponding to clustering results whose ratio of the number of categories to the number of clustered data is greater than a preset ratio, and / or initial parameter combinations corresponding to clustering results whose proportion of the number of valid sample points in the clustered data is not within a preset proportion range.

[0017] Further, determining the intent category according to the clustering result of the clustering model under the clustering data and the optimal parameter combination includes:

[0018] Inputting the sentence vector corresponding to the clustering data into the clustering model under the optimal parameter combination for clustering, so as to determine at least one initial cluster center point corresponding to the clustering result;

[0019] Preprocessing the clustering data according to preset rules to generate intent data, wherein the preprocessing includes data enhancement and data amplification;

[0020] Optimizing each of the initial cluster centers according to the sentence vector corresponding to the intent data to determine the corresponding category cluster center;

[0021] The intent category is determined based on the center point of each category cluster and the corresponding user sentence.

[0022] Furthermore, the obtaining of training data and clustering data includes:

[0023] Get historical conversation data;

[0024] The historical conversation data is preprocessed to determine training data and clustering data from the preprocessed historical conversation data.

[0025] Furthermore, performing comparative learning training on the pre-trained language model according to the training data to determine the sentence vector model includes:

[0026] Constructing positive sample pairs and negative sample pairs according to the training data, wherein the positive sample pairs are user sentences and dialogue sentences with contextual relationships in the same training data, and the negative sample pairs are user sentences and / or dialogue sentences without contextual relationships;

[0027] Determine a loss function corresponding to the pre-trained language model according to the positive sample pairs and the negative sample pairs;

[0028] The pre-trained language model is trained by contrastive learning according to the positive sample pairs, negative sample pairs and loss function to determine a sentence vector model.

[0029] Furthermore, the step of inputting the clustering data into the sentence vector model to determine the corresponding sentence vector includes:

[0030] Inputting the clustering data into the sentence vector model for processing;

[0031] An average operation is performed on all vectors corresponding to the last layer of the sentence vector model to determine the corresponding sentence vector.

[0032] Furthermore, the method further comprises:

[0033] The intention category is saved in a database to output corresponding conversation data according to the intention category.

[0034] Furthermore, the method further comprises:

[0035] Performing a similarity test between the intent category and the existing intent categories to determine a new intent category;

[0036] The newly added intent category is saved in a database to output corresponding dialogue data according to the newly added intent category and the existing intent category.

[0037] In a second aspect, an embodiment of the present invention is intended to provide an intention recognition device, the device comprising:

[0038] A data acquisition unit, used to acquire training data and clustering data, wherein the training data includes user sentences and dialogue sentences, and the clustering data includes user sentences;

[0039] A model determination unit, configured to perform comparative learning training on the pre-trained language model according to the training data to determine a sentence vector model;

[0040] A vector determination unit, used for inputting the clustering data into the sentence vector model to determine a corresponding sentence vector;

[0041] A parameter determination unit, used for inputting the sentence vectors into the clustering models corresponding to the initial parameter combinations for clustering, so as to determine the best parameter combination according to the clustering results;

[0042] The category determination unit is used to determine the intent category according to the clustering result of the clustering model under the clustering data and the optimal parameter combination.

[0043] In a third aspect, an embodiment of the present invention aims to provide a computer program product, wherein the computer program product comprises a computer program / instructions, and when the computer program / instructions are executed by a processor, the method as described in any one of the above items is implemented.

[0044] In a fourth aspect, an embodiment of the present invention aims to provide an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement a method as described in any one of the above items.

[0045] In a fifth aspect, an embodiment of the present invention aims to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps as described in any one of the above items are implemented.

[0046] The technical solution of the embodiment of the present invention obtains training data and clustering data, performs comparative learning training on the pre-trained language model according to the training data, determines the sentence vector model, inputs the clustering data into the sentence vector model, determines the corresponding sentence vector, and inputs the sentence vector into the clustering model corresponding to each initial parameter combination for clustering, so as to determine the best parameter combination according to each clustering result, and determine the intent category according to the clustering result of the clustering model under the clustering data and the best parameter combination. Therefore, by determining the sentence vector model through comparative learning training and determining the best clustering parameter combination and the corresponding intent category according to the clustering results corresponding to different initial parameter combinations, the accuracy of new intent recognition can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0048] Figure 1 is a flow chart of an intention recognition method according to an embodiment of the present invention;

[0049] Figure 2 is a flowchart of a method for determining a sentence vector model according to an embodiment of the present invention;

[0050] Figure 3 is a flowchart of a method for determining a sentence vector according to an embodiment of the present invention;

[0051] Figure 4 is a flow chart of a method for determining an optimal parameter combination according to an embodiment of the present invention;

[0052] Figure 5 is a flow chart of a method for determining an intent category according to an embodiment of the present invention;

[0053] Figure 6 is another flow chart of the intention recognition method according to an embodiment of the present invention;

[0054] Figure 7 is a schematic diagram of an intention recognition device according to an embodiment of the present invention;

[0055] Figure 8 is another schematic diagram of the intention recognition device according to an embodiment of the present invention;

[0056] Fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the detailed description of the present invention below, some specific details are described in detail. It is possible for a person skilled in the art to fully understand the present invention without the description of these details. In order to avoid confusing the essence of the present invention, known methods, processes, flows, components and circuits are not described in detail.

[0058] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0059] Unless the context clearly requires otherwise, the words "include", "comprising" and the like throughout this application should be interpreted as including rather than exclusive or exhaustive; that is, as meaning "including but not limited to".

[0060] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0061] The solutions described in this specification and in the embodiments, if they involve the processing of personal information (including user data, etc.), will be processed (including collection, storage, transmission and use, etc.) on the premise of having a legal basis (such as obtaining the consent of the subject of personal information, complying with the provisions of laws and administrative regulations, or being necessary for the performance of a contract, etc.), and will only be processed within the scope of regulations or agreements. The user's refusal to process personal information other than the necessary information for basic functions will not affect the user's use of basic functions.

[0062] In the prior art, since vectorization tools are directly used to vectorize user questions and then traditional machine learning clustering algorithms are used to cluster user questions to realize user intent recognition, there is a problem of inaccurate intent recognition. The embodiment of the present invention aims to improve the accuracy of intent recognition through the following intent recognition method.

[0063] In this embodiment, the intent recognition method applied to the intelligent customer service system is used as an example for explanation, so that after the user asks a question, a suitable reply can be given according to the intent corresponding to the user's question, thereby providing the user with a better user experience. It should be understood that the intent recognition method in this embodiment can also be applied to other dialogue task systems or intent recognition scenarios, and this is not limited here.

[0064] Figure 1 is a flow chart of the intention recognition method of an embodiment of the present invention. Figure 1 As shown, the intention recognition method of this embodiment includes the following steps.

[0065] In step S100, training data and clustering data are obtained.

[0066] In this embodiment, the training data includes user sentences and dialogue sentences. The user sentences correspond to the user questions input by the user, and the dialogue sentences correspond to the responses made by the customer service staff to the user questions. The training data is used to determine the sentence vector model so that the user sentences corresponding to the user questions and the dialogue sentences in response to the user questions can be vectorized using the sentence vector model. The clustering data includes user sentences, which are used to cluster and determine the user intent.

[0067] Optionally, the training data and clustering data in this embodiment can be determined by processing historical dialogue data. The historical dialogue data can be a dialogue log between a user and a manual customer service or an intelligent customer service. In the cold start phase before the intelligent customer service system goes online, the historical dialogue data includes user statements input by the user and dialogue statements between the manual customer service. In the operation phase after the intelligent customer service system goes online, the historical dialogue data includes user statements within the historical operation time period of the intelligent customer service system and dialogue statements output by the intelligent customer service system for user statements. Among them, each dialogue log may contain multiple user statements corresponding to user questions and dialogue statements given by the customer service for each user question. The specific format of the dialogue log includes one or more user statements and one or more dialogue statements. Each user statement and dialogue statement has a corresponding role identifier to distinguish whether it is a statement corresponding to the user or the customer service.

[0068] Furthermore, in this embodiment, when determining training data and clustering data based on historical conversation data, the historical conversation data is first obtained, and the historical conversation data is preprocessed to determine training data and clustering data from the preprocessed historical conversation data. Thus, by effectively utilizing the existing massive historical conversation data for intent recognition and mining, effective intent data can be provided for the construction and launch of the intelligent customer service system and the optimization of operations after launch, which can reduce the heavy work of manually mining intent data and improve the efficiency of intent recognition.

[0069] Optionally, preprocessing of the conversation data includes filtering out data without obvious intent and other forms of interference data such as non-Chinese data, such as hello, thank you, goodbye, abusive words, etc. By preprocessing the conversation data and filtering out the interference data in the conversation data, the quality of the training data and clustering data can be improved, thereby improving the accuracy and efficiency of subsequent intent recognition.

[0070] Furthermore, when determining the training data, the preprocessing of the conversation data may also include constructing the sentences corresponding to two adjacent roles in the historical conversation log into one piece of training data, and separating the user sentences and the customer service conversation sentences in the same piece of training data with the line break key. Assuming that there are multiple sentences corresponding to the same role, the multiple sentences are concatenated, for example: User sentence: How do I get an invoice for taking a taxi? Conversation sentence: You can use the mobile phone APP, click... When determining the clustering data in the cold start phase, the preprocessing of the conversation data may also include a deduplication operation, in which repeated questions raised by users are deduplicated to ensure that there is no exactly the same data in the clustering data used for intent recognition; when determining the clustering data after the system is put into operation, the preprocessing of the conversation data includes screening user data without obvious intent, so as to identify the intent of the screened clustering data and determine the intent category corresponding to the user question that the system cannot recognize.

[0071] In step S200, a pre-trained language model is subjected to comparative learning training according to the training data to determine a sentence vector model.

[0072] In this embodiment, vectorizing the sentences corresponding to the clustering data is an important step in intent recognition. Since the existing vectorization expression tools cannot meet the requirements of the clustering method for sentence vectorization accuracy, in this embodiment, the vector model is determined by comparing the pre-trained language model with the training data, and the sentence vector model is determined by self-supervised learning using the context of the conversation text in the historical conversation data as the training corpus. The sentence vector determined according to the sentence vector model can better serve the subsequent clustering and intent recognition, which is conducive to improving the accuracy of intent recognition.

[0073] Optionally, in this embodiment, an existing pre-trained language model may be used for training to determine a sentence vector model, and preferably the Roberta-wwm-ext-large model is used as a benchmark model for training to determine a corresponding sentence vector model. Figure 2 The method shown is used to determine the sentence vector model.

[0074] In step S210, positive sample pairs and negative sample pairs are constructed according to the training data.

[0075] In this embodiment, a piece of training data consists of a user statement about the same question and a conversation statement of the corresponding customer service, and there is a contextual relationship between the user statement and the conversation statement in the same piece of training data. Multiple pieces of training data are used when determining the sentence vector model, and the more pieces of training data there are, the higher the accuracy of the determined sentence vector model. At the same time, in this embodiment, the positive sample pairs constructed based on the training data are user statements and conversation statements with contextual relationships in the same piece of training data, and the negative sample pairs are user statements and / or conversation statements without contextual relationships.

[0076] In step S220, a loss function corresponding to the pre-trained language model is determined according to the positive sample pairs and the negative sample pairs.

[0077] In this embodiment, the loss function corresponding to the pre-trained language model is used to make the positive sample pairs after training close to each other and the negative sample pairs far away from each other. Assume that the training data corresponding to a training process (corresponding to a batch) is composed of multiple training data, including {data1, data2, ···, datan}, where data1 is composed of user statement utterance1 and customer service dialogue statement utterance2. When determining the loss function, first calculate and determine the cosine similarity of the user statement utterance1 and the dialogue statement utterance2 corresponding to the same training data as the numerator, then calculate and determine the sum of the cosine similarities of the user statement utterance1 in the current training data and the negative samples constructed by other n-1 training data as the denominator, and divide the numerator by the denominator to calculate the negative logarithm of the corresponding ratio; then calculate and determine the cosine similarity of the dialogue statement utterance2 and the user statement utterance1 in the same training data as the numerator, then calculate the cosine similarity of the dialogue statement utterance2 in the current training data and the negative samples constructed by other n-1 data as the denominator, and divide the numerator by the denominator to calculate the negative logarithm of the corresponding ratio. Finally, the negative logarithmic values ​​of the above two parts are added together to obtain the loss function value of a single training data in the current training process batch, and the loss function of the entire batch, that is, the loss function of all training data, is added together and then divided by twice the batch sample size. Thus, the loss function of the pre-trained language model is determined through the above process.

[0078] In step S230, the pre-trained language model is trained by contrastive learning based on the positive sample pairs, the negative sample pairs and the loss function to determine a sentence vector model.

[0079] In this embodiment, after determining the positive sample pairs, negative sample pairs and loss functions corresponding to the training data, the pre-trained language model is trained by a comparative learning method, which can make the semantics of adjacent sentences (including user sentences and dialogue sentences) in the same dialogue (corresponding to a piece of training data) closer, and make the semantics of adjacent sentences in different dialogues more distant, so that the determined sentence vector model can more effectively represent the vector features of the sentence, reduce the error caused by the subsequent intent recognition process, and improve the accuracy of intent recognition.

[0080] In step S300, the clustered data is input into a sentence vector model to determine a corresponding sentence vector.

[0081] In this embodiment, after determining the sentence vector model, Figure 3 The method shown is used to determine the sentence vector corresponding to the clustered data.

[0082] In step S310, the clustered data is input into the sentence vector model for processing.

[0083] In this embodiment, the cluster data determined from the historical conversation data is input into the determined sentence vector model, the cluster data is processed by the sentence vector model, and the sentence vector corresponding to the cluster data is output.

[0084] In step S320, all vectors corresponding to the last layer of the sentence vector model are averaged to determine the sentence vector corresponding to the clustered data.

[0085] In this embodiment, the operation of averaging all vectors corresponding to the last layer of the sentence vector model refers to averaging all word vectors in the sentence to ensure that the number of words corresponding to the input sentence before and after model processing is consistent, which is beneficial to improving the accuracy of determining the sentence vector corresponding to the clustering data and providing convenience for subsequently improving the accuracy of intent recognition.

[0086] In step S400, the sentence vectors are respectively input into the clustering models corresponding to the initial parameter combinations for clustering, so as to determine the best parameter combination according to the clustering results.

[0087] In this embodiment, after determining the sentence vectors corresponding to the clustered data, the sentence vectors corresponding to the clustered data are clustered by the clustering model to determine the intent category. Since the number of real intent categories corresponding to the clustered data cannot be determined in advance, this embodiment uses a density-based clustering algorithm to determine the number of intent categories, and determines the intent category by the distribution of valid sample points corresponding to the clusters.

[0088] Optionally, the clustering model in this embodiment can adopt various forms of clustering models, preferably the HDBSCAN model. Since the parameters that affect the clustering results mainly include min cluster size, min samples, and cluster selection epsilon, and the clustering results obtained by different parameter combinations are different, when performing intent recognition, this embodiment will input the sentence vector corresponding to the clustering data into the clustering model corresponding to different initial parameter combinations, and determine the optimal parameter combination based on the corresponding clustering results under different initial parameter combinations.

[0089] Optionally, in this embodiment, when determining the initial parameter combination, the parameter space corresponding to each clustering parameter is traversed in turn to determine the initial parameter combination. The parameter space includes at least one optional value corresponding to the clustering parameter. Further, the parameter space corresponding to the clustering parameter min cluster size in this embodiment is {3, 4, 5, 6, 7, 8, 9, 10}, including 8 optional values; the parameter space corresponding to the clustering parameter min samples is {2, 3, 4, 5, 6, 7, 8, 9, 10}, including 9 optional values; the parameter space corresponding to the clustering parameter cluster selection epsilon is {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}, including 9 optional values. Therefore, by combining the optional values ​​of the above different parameters, 8*9*9 initial parameter combinations can be determined, and the corresponding number of clustering results can be obtained.

[0090] It should be noted that the optional values ​​of each clustering parameter and the corresponding parameter space in this embodiment are determined by pre-accumulated model usage experience and can be set and adjusted in different application scenarios. This is only an example and not a limitation.

[0091] Since the clustering model will output different clustering results under different initial parameter combinations, the clustering results are usually composed of outlier sample points and valid sample points, where outliers are also called outliers, corresponding to points with a clustering result label of -1, representing sample points whose values ​​in the sample are significantly deviated from the remaining values. Valid sample points correspond to points whose clustering result labels are not -1, and valid sample points that meet certain similar characteristics form a cluster, one cluster corresponds to one category, and the distribution of valid sample points can reflect the number of categories under the corresponding initial parameter combination. Therefore, the clustering results in this embodiment include outlier sample points, valid sample points and the number of categories, so as to determine the optimal parameter combination according to each clustering result under the clustering model corresponding to each initial parameter combination.

[0092] Alternatively, if Figure 4 As shown, in this embodiment, the optimal parameter combination is determined according to each clustering result by the following method.

[0093] In step S410, each initial parameter combination is screened based on a preset screening rule and a clustering result corresponding to each initial parameter combination to determine at least one candidate parameter combination.

[0094] Optionally, the screening rules in this embodiment are used to screen out the initial parameter combinations corresponding to the clustering results whose number of categories is less than a predetermined number, the initial parameter combinations corresponding to the clustering results whose ratio of the number of categories to the number of clustered data is greater than a preset ratio, and the initial parameter combinations corresponding to the clustering results whose proportion of the number of valid sample points in the clustered data is not within a preset weight interval. For example, in this embodiment, the predetermined number of categories is set to 3, the preset ratio is set to 50%, and the preset weight interval is set to 30%-50%. In this embodiment, the initial parameter combinations corresponding to the clustering results including the following situations are determined as unreasonable parameter combinations, including the initial parameter combination with the number of cluster categories less than 3, the number of cluster categories greater than 50% of the number of valid sample points, the proportion of the number of valid sample points in the total data volume corresponding to the clustered data is less than 30%, and the proportion of the number of valid sample points in the total data volume corresponding to the clustered data is greater than 50%. After determining the unreasonable initial parameter combinations, the remaining reasonable initial parameter combinations are determined as candidate parameter combinations to determine the best parameter combination based on the candidate parameter combinations.

[0095] In step S420, the candidate parameter combination corresponding to the clustering result with the largest number of categories is determined as the optimal parameter combination.

[0096] In this embodiment, after determining the candidate parameter combinations, the optimal parameter combination of the clustering model is determined according to the clustering results corresponding to each candidate parameter combination. Optionally, in this embodiment, the candidate parameter combination corresponding to the clustering result with the largest number of categories is determined as the optimal parameter combination. When the clustering result with the largest number of categories corresponds to different candidate parameter combinations, the candidate parameter combinations can continue to be clustered until the clustering result with the largest number of categories corresponds to a parameter combination, and the parameter combination is determined as the optimal parameter combination; the candidate parameter combinations corresponding to multiple clustering results with the largest number of categories can also be retained as the optimal parameter combination at the same time to improve the accuracy of the optimal parameter combination; any one of the candidate parameter combinations corresponding to the clustering results with the largest number of categories can also be selected as the optimal parameter combination to improve the efficiency of determining the optimal parameter combination, which is conducive to improving the efficiency of subsequent intent recognition.

[0097] Furthermore, after determining the optimal parameter combination corresponding to the clustering model, this embodiment inputs the clustering data into the clustering result output by the clustering model corresponding to the optimal parameter combination, and determines the number of clusters to which the valid sample points belong as the number of intent categories of the intent to be identified.

[0098] In step S500, the intent category is determined according to the clustering result of the clustering model under the clustering data and the optimal parameter combination.

[0099] In this embodiment, in order to facilitate the rapid determination of the user intent category and the dialogue sentences corresponding to the user intent, it is necessary to determine the central sentence vector under each intent category and the intent sentence corresponding to each intent category after determining the number of intent categories corresponding to the clustering data, that is, to determine the intent category. Optionally, in this embodiment, the intent category can be determined directly based on the clustering results of the clustering data and the clustering model under the optimal parameter combination, or the intent data can be determined by further processing the clustering data, and the intent category can be determined based on the intent data. Furthermore, in order to improve the accuracy of intent recognition and the richness of intent category content, the method of generating intent data is used in this embodiment to determine the intent category.

[0100] Figure 5 FIG. 1 is a flow chart of a method for determining an intent category according to an embodiment of the present invention. Figure 5 As shown, in this embodiment, the intent category is determined through the following steps.

[0101] In step S510, the sentence vector corresponding to the clustering data is input into the clustering model under the optimal parameter combination for clustering to determine at least one initial cluster center point corresponding to the clustering result.

[0102] In this embodiment, the sentence vectors corresponding to the clustering data are input into the clustering model under the optimal parameter combination for clustering to determine the clustering result. In the clustering result, the valid sample points with high similarity are concentrated to form corresponding category clusters, and each category cluster has a corresponding initial cluster center point, and each initial cluster center point has a corresponding sentence vector.

[0103] In step S520, the cluster data is preprocessed according to preset rules to generate intent data, wherein the preprocessing includes data enhancement and data amplification.

[0104] In this embodiment, in order to enrich the category data, the cluster data is amplified or enhanced according to the preset rules in this implementation to generate a larger amount of intent data, so that the intent categories generated based on the intent data can cover more content. Specifically, it is assumed that the intent data includes multiple data {data1', data2', ···, datan'}, and each data datan' is composed of an original cluster data datan and the intent data datan.1, datan.2, etc. obtained after the cluster data is amplified.

[0105] Optionally, in this embodiment, data with the same text meaning can be constructed according to each cluster data to perform data augmentation; or a certain number of words (such as 20% of the words) can be randomly deleted for each cluster data, and it is ensured that the intention expressed by the sentence formed after deleting the words does not change, so as to achieve the effect of data enhancement. For example, the original text is "Today, after I got off the bus, I left my mobile phone and wallet in the bus. I want to contact the driver as soon as possible to retrieve my mobile phone and wallet." The text formed after the original text data enhancement is "After getting off the bus, my mobile phone and wallet were left in the bus. I want to contact the driver to find my mobile phone and wallet." Thus, by performing data augmentation or data enhancement on the cluster data, a larger amount of intent data is generated, thereby enriching the intent categories generated based on the intent data.

[0106] In step S530, each initial cluster center point is optimized according to the sentence vector corresponding to the intent data to determine the corresponding category cluster center point.

[0107] In this embodiment, the initial cluster center is determined based on the clustering results corresponding to the clustering data in the historical conversation data, and the number of initial cluster centers is consistent with the number of cluster categories in the clustering results under the optimal parameter combination. After determining the initial cluster center, since the intent data used to identify the intent category is expanded compared to the clustering data, in order to ensure the accuracy of intent recognition, in this embodiment, the initial cluster center is optimized based on the intent data to determine the category cluster center point.

[0108] Optionally, after determining the number of intent categories, this embodiment first inputs the sentence corresponding to the intent data into the sentence vector model determined above, and then inputs the sentence vector corresponding to the intent data into the K-MEANS clustering model for clustering, and then optimizes the initial cluster center point to determine the category cluster center point corresponding to the intent data and the intent sentences (i.e., user sentences) corresponding to each category cluster.

[0109] Specifically, in this embodiment, two optimization tasks are involved in the process of clustering through the K-MEANS clustering model. One is to use KL divergence to iteratively optimize the cluster distribution of each sample point in the intent data, and the other is to use contrastive learning to iteratively optimize the sentence vector of the clustered sentence. When optimizing the cluster distribution of the sample points corresponding to the intent data, the learning t distribution is used to calculate the allocation probability of data1.1 to the kth category. According to the initialized cluster center point, a target distribution can be calculated, and the KL divergence is used to shorten the distance between the allocation probability of each sample point and the target distribution. At the same time, when using contrastive learning to iteratively optimize the sentence vector of the clustered sentence, positive and negative sample pairs are constructed based on the amplified part of the data: the positive sample pair is composed of data1.1 and data1.2, and the negative sample pair is composed of data1.1 and other sample points in the same batch. The contrastive learning loss function is also used for optimization.

[0110] In step S540, the intent category is determined based on the center point of each category cluster and the corresponding user sentence.

[0111] In this embodiment, after the intent data is clustered, the category cluster center of each cluster in the clustering result corresponding to the intent data and the user sentence corresponding to each category cluster are determined as the final recognized intent category.

[0112] Optionally, after determining the intent category after the cold start phase, this embodiment will save the identified intent recognition into the database to output corresponding dialogue data according to the identified intent category. Alternatively, after determining the intent category in the operation phase after the system goes online, the identified intent category will be tested for similarity with the existing categories in the intelligent customer service system to determine the newly added intent category, and the newly added intent category will be saved into the database to output corresponding dialogue data according to the newly added intent category and the existing intent category.

[0113] The technical solution of the embodiment of the present invention obtains training data and clustering data, performs comparative learning training on the pre-trained language model according to the training data, determines the sentence vector model, inputs the clustering data into the sentence vector model, determines the corresponding sentence vector, and inputs the sentence vector into the clustering model corresponding to each initial parameter combination for clustering, so as to determine the best parameter combination according to each clustering result, and determine the intent category according to the clustering results of the clustering model under the clustering data and the best parameter combination. Thus, by determining the sentence vector model through comparative learning training and determining the best parameter combination for clustering and identifying the intent category according to the clustering results corresponding to different initial parameter combinations, the accuracy of new intent recognition can be improved.

[0114] Figure 6 FIG. 4 is another flow chart of the method for identifying intentions according to an embodiment of the present invention. Figure 6 As shown, the intention recognition method of this embodiment includes the following steps.

[0115] In step S610, historical conversation data is obtained.

[0116] In this embodiment, the historical conversation statements can be the conversation logs between users and manual customer service before the artificial intelligence customer service goes online, or they can be the conversation logs between users and intelligent customer service collected after the artificial intelligence customer service system goes online.

[0117] In step S620, training data and clustering data are determined based on the historical conversation data.

[0118] In this embodiment, training data and clustering data are determined by processing historical conversation data. Optionally, after obtaining historical conversation data, the sentences corresponding to two adjacent roles (user and customer service) in a conversation log are used as a piece of training data, and the line break key is used to separate different roles, and multiple consecutive sentences of the same role are spliced. At the same time, the sentences obtained after deduplication of user sentences in the conversation log are used as clustering data.

[0119] In step S630, a sentence vector model is determined according to the training data.

[0120] In this embodiment, positive sample pairs and negative sample pairs are constructed according to the training data, and then the loss function corresponding to the pre-trained language model is determined according to the positive sample pairs and the negative samples. Finally, the pre-trained language model is trained by comparative learning through the positive sample pairs and the negative samples and the determined loss function to determine the sentence vector model so as to vectorize the input sentence through the sentence vector model.

[0121] In step S640, the clustered data is input into a sentence vector model to determine a sentence vector corresponding to the clustered data.

[0122] In step S650, the sentence vectors corresponding to the clustering data are input into the clustering models corresponding to each initial parameter combination for clustering to determine the best parameter combination.

[0123] In this embodiment, before clustering begins, the corresponding clustering parameters and the parameter space corresponding to at least one optional value corresponding to each clustering parameter are determined based on experience and intention recognition scenarios, and the parameter space of each clustering parameter is traversed in turn to determine each initial parameter combination, and the sentence vector corresponding to the clustering data is input into the clustering model corresponding to different initial parameter combinations for clustering to determine the clustering results. The clustering results under different initial parameter combinations are screened by preset screening rules, unreasonable parameter combinations are removed, at least one candidate parameter combination is determined, and the candidate parameter combination corresponding to the clustering result with the largest number of categories is determined as the optimal parameter combination.

[0124] In step S660, the number of cluster categories is determined according to the clustering result corresponding to the clustering model under the optimal parameter combination.

[0125] In this embodiment, the clustering results corresponding to the clustering model under the optimal parameter combination include multiple clusters, each cluster corresponds to a clustering category, and thus the number of clusters in the clustering results corresponding to the clustering model under the optimal parameter combination is determined as the number of clustering categories during intent recognition.

[0126] In step S670, intention data is generated based on the clustering data.

[0127] In this embodiment, intent data is constructed and generated by performing data enhancement and data amplification on cluster data according to preset rules so that the intent data can include richer user sentences, and thus the intent categories generated based on the intent data can cover more content, which is conducive to improving the accuracy of intent recognition.

[0128] In step S680, clustering is performed based on the number of cluster categories and the intent data to determine the intent category.

[0129] In this embodiment, after clustering the cluster data using the clustering model under the optimal parameter combination, the number of cluster categories, at least one initial cluster center point corresponding to the cluster data, and the sentence vector corresponding to each initial cluster center point can be determined based on the clustering results. After determining the number of intent categories, each initial cluster center point is optimized based on the sentence vector corresponding to the intent data, and the corresponding category cluster center point and the user sentence corresponding to each category cluster are determined, and then the category cluster center point corresponding to the intent data and the user sentence corresponding to each category cluster are determined as the intent category to be identified.

[0130] In step S690, the intent categories are compared and stored.

[0131] In this embodiment, when the determined intent category is used for the launch of the intelligent customer service system, the identified intent recognition will be saved in the database to output the corresponding dialogue data according to the identified intent category. When the determined intent recognition is used to optimize the existing intelligent customer service system, the identified intent category will first be tested for similarity with the existing intent categories in the intelligent customer service system, and a new intent category will be determined, and the new intent category will be saved in the database to output the corresponding dialogue data according to the new intent category and the existing intent category.

[0132] The technical solution of the embodiment of the present invention obtains training data and clustering data, performs comparative learning training on the pre-trained language model according to the training data, determines the sentence vector model, inputs the clustering data into the sentence vector model, determines the corresponding sentence vector, and inputs the sentence vector into the clustering model corresponding to each initial parameter combination for clustering, so as to determine the best parameter combination according to each clustering result, and determines the intent category according to the clustering result of the clustering model under the clustering data and the best parameter combination. Thus, by determining the sentence vector model through comparative learning training and determining the best clustering parameter combination and identifying the intent category according to the clustering results corresponding to different initial parameter combinations, the accuracy of new intent recognition can be improved. At the same time, the above method can effectively utilize the existing massive historical conversation data for intent recognition and mining, can provide effective intent data for the construction and launch of intelligent customer service systems and the optimization of operations after launch, can reduce the heavy work of manually mining intent data, and improve the efficiency of intent recognition.

[0133] Figure 7 Schematic diagram of an intention recognition device according to an embodiment of the present invention. Figure 7 As shown, the intention recognition device of this embodiment includes a data acquisition unit 1, a model determination unit 2, a vector determination unit 3, a parameter determination unit 4 and a category determination unit 5. Among them, the data acquisition unit 1 is used to obtain training data and clustering data. The model determination unit 2 is used to perform comparative learning training on the pre-trained language model according to the training data to determine the sentence vector model. The vector determination unit 3 is used to input the clustering data into the sentence vector model to determine the corresponding sentence vector. The parameter determination unit 4 is used to input the sentence vectors into the clustering models corresponding to each initial parameter combination for clustering, so as to determine the best parameter combination according to each clustering result. The category determination unit 5 is used to determine the intent category according to the clustering results of the clustering model under the clustering data and the best parameter combination.

[0134] Figure 8 FIG. 1 is another schematic diagram of the intention recognition device according to an embodiment of the present invention. Figure 8 As shown, the data acquisition unit 1 in the intention recognition device of this embodiment includes an acquisition module 11 and a processing module 12. The acquisition module 11 is used to acquire historical dialogue data. The processing module 12 is used to pre-process the historical dialogue data to determine training data and clustering data from the pre-processed historical dialogue data. The training data includes user sentences and dialogue sentences, and the clustering data includes user sentences.

[0135] Optionally, the model determination unit 2 in this embodiment includes a construction module 21, a determination module 22 and a training module 23. Among them, the construction module 21 is used to construct positive sample pairs and negative sample pairs according to the training data. Among them, the positive sample pairs are user sentences and dialogue sentences with contextual relationships in the same training data, and the negative sample pairs are user sentences and / or dialogue sentences without contextual relationships. The determination module 22 is used to determine the loss function corresponding to the pre-trained language model based on the positive sample pairs and the negative sample pairs. The training module 23 is used to perform comparative learning training on the pre-trained language model based on the positive sample pairs, the negative sample pairs and the loss function to determine the sentence vector model.

[0136] Optionally, the vector determination unit 3 in this embodiment is specifically used to input the clustering data into the sentence vector model for processing, and to average all vectors corresponding to the last layer of the sentence vector model to determine the corresponding sentence vector. The parameter determination unit 4 in this embodiment is used to input the sentence vectors into the clustering models corresponding to each initial parameter combination for clustering, so as to determine the best parameter combination according to each clustering result. The category determination unit 5 in this embodiment is used to determine the intent category according to the clustering results of the clustering model under the clustering data and the best parameter combination.

[0137] Optionally, the intent recognition device in this embodiment further includes a category storage unit 6. The category storage unit 6 is used to store the identified intent category in a database, so as to output corresponding dialogue sentences according to the intent category after the user inputs dialogue data. Furthermore, the category storage unit 6 is specifically used to store the intent category in a database during the cold start phase, so as to output corresponding dialogue data according to the intent category, or to perform a similarity check between the identified intent category and the existing intent category during the operation phase, determine a newly added intent category, and store the newly added intent category in a database, so as to output corresponding dialogue data according to the newly added intent category and the existing intent category.

[0138] Fig. 9 Schematic diagram of an electronic device according to an embodiment of the present invention. Fig. 9 As shown, Fig. 9 The electronic device shown is a general address query device, which includes a general computer hardware structure, which at least includes a processor 91 and a memory 92. The processor 91 and the memory 92 are connected via a bus 93. The memory 92 is suitable for storing instructions or programs executable by the processor 91. The processor 91 can be an independent microprocessor or a collection of one or more microprocessors. Thus, the processor 91 executes the instructions stored in the memory 92, thereby executing the method flow of the embodiment of the present invention as described above to realize the processing of data and the control of other devices. The bus 93 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to the display controller 94 and the display device and the input / output (I / O) device 95. The input / output (I / O) device 95 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices known in the art. Typically, the input / output device 95 is connected to the system via an input / output (I / O) controller 96.

[0139] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices (equipment) or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] The present application is described with reference to flowcharts of methods, apparatuses (devices) and computer program products according to embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0141] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device that implements the process Figure 1 A function specified in a process or multiple processes.

[0142] These computer program instructions may also be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce the instructions for implementing the process Figure 1 A device that specifies functions in a process or multiple processes.

[0143] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used for a computer to execute part or all of the above method embodiments.

[0144] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by specifying relevant hardware through a program, and the program is stored in a storage medium, including several instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0145] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for intention recognition, characterized in that: The method comprises: Acquire training data and clustering data, wherein the training data includes user sentences and dialogue sentences, and the clustering data includes user sentences; Performing comparative learning training on the pre-trained language model according to the training data to determine a sentence vector model; Inputting the clustering data into the sentence vector model to determine the corresponding sentence vector; Inputting the sentence vectors into the clustering models corresponding to the initial parameter combinations for clustering, so as to determine the best parameter combination according to the clustering results; The intent category is determined according to the clustering data and the clustering result of the clustering model under the optimal parameter combination.

2. The method according to claim 1, characterized in that The initial parameter combination is determined based on the following steps: The parameter space corresponding to each clustering parameter is traversed in turn to determine an initial parameter combination, wherein the parameter space includes at least one optional value corresponding to the clustering parameter.

3. The method according to claim 1, characterized in that The clustering results include outlier sample points, valid sample points and the number of categories, and determining the best parameter combination according to each of the clustering results includes: Screening the initial parameter combinations based on preset screening rules and clustering results corresponding to the initial parameter combinations to determine at least one candidate parameter combination; The candidate parameter combination corresponding to the clustering result with the largest number of categories is determined as the optimal parameter combination.

4. The method according to claim 3, characterized in that The screening rules are used to screen out initial parameter combinations corresponding to clustering results whose number of categories is less than a predetermined number, initial parameter combinations corresponding to clustering results whose ratio of the number of categories to the number of clustered data is greater than a preset ratio, and / or initial parameter combinations corresponding to clustering results whose proportion of the number of valid sample points in the clustered data is not within a preset proportion range.

5. The method according to claim 1, characterized in that Determining the intent category according to the clustering result of the clustering model under the clustering data and the optimal parameter combination includes: Inputting the sentence vector corresponding to the clustering data into the clustering model under the optimal parameter combination for clustering, so as to determine at least one initial cluster center point corresponding to the clustering result; Preprocessing the clustering data according to preset rules to generate intent data, wherein the preprocessing includes data enhancement and data amplification; Optimizing each of the initial cluster centers according to the sentence vector corresponding to the intent data to determine the center of the category cluster; The intent category is determined according to the category cluster center point and the user sentence corresponding to the category cluster center point.

6. The method according to claim 1, characterized in that The obtaining of training data and clustering data comprises: Get historical conversation data; The historical conversation data is preprocessed to determine training data and clustering data from the preprocessed historical conversation data.

7. The method according to claim 1, characterized in that The performing comparative learning training on the pre-trained language model according to the training data to determine the sentence vector model includes: Constructing positive sample pairs and negative sample pairs according to the training data, wherein the positive sample pairs are user sentences and dialogue sentences with contextual relationships in the same training data, and the negative sample pairs are user sentences and / or dialogue sentences without contextual relationships; Determine a loss function corresponding to the pre-trained language model according to the positive sample pairs and the negative sample pairs; The pre-trained language model is trained by contrastive learning according to the positive sample pairs, negative sample pairs and loss function to determine a sentence vector model.

8. The method according to claim 1, characterized in that The step of inputting the clustering data into the sentence vector model to determine the corresponding sentence vector includes: Inputting the clustering data into the sentence vector model for processing; An average operation is performed on all vectors corresponding to the last layer of the sentence vector model to determine the corresponding sentence vector.

9. The method according to claim 1, characterized in that: The method further comprises: The intention category is saved in a database to output corresponding conversation data according to the intention category.

10. The method according to claim 1, characterized in that The method further comprises: Performing a similarity test between the intent category and the existing intent categories to determine a new intent category; The newly added intent category is saved in a database to output corresponding conversation data according to the newly added intent category and the existing intent category.

11. An intention recognition device, characterized in that: The device comprises: A data acquisition unit, used to acquire training data and clustering data, wherein the training data includes user sentences and dialogue sentences, and the clustering data includes user sentences; A model determination unit, configured to perform comparative learning training on the pre-trained language model according to the training data to determine a sentence vector model; A vector determination unit, used for inputting the clustering data into the sentence vector model to determine a corresponding sentence vector; A parameter determination unit, used for inputting the sentence vectors into the clustering models corresponding to the initial parameter combinations for clustering, so as to determine the best parameter combination according to the clustering results; The category determination unit is used to determine the intent category according to the clustering result of the clustering model under the clustering data and the optimal parameter combination.

12. A computer program product, characterized in that The computer program product comprises a computer program / instructions, and when the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

13. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 10 are implemented.

Images