Model training, intelligent customer service question clustering method and related apparatus

Abstract

The application relates to the technical field of artificial intelligence, and provides a model training, intelligent customer service question clustering method and related devices, the method comprising: inputting a training set into a clustering feature representation model to obtain a feature vector of each training data, the training set comprising labeled training data and unlabeled training data; clustering each feature vector to obtain clustering clusters and a cluster label corresponding to each clustering cluster; when the clustering clusters do not satisfy a clustering condition, iteratively optimizing each cluster label according to a label set and / or each cluster label, the label set comprising a real label of each labeled training data in the training set; and training the clustering feature representation model based on the training data and the optimized cluster label to obtain a trained clustering feature representation model. The application adjusts the initial cluster label by using the real label, and then trains the model by using the adjusted cluster label, so that the clustering effect of the trained clustering feature representation model is better.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a model training and intelligent customer service question clustering method and related device. BACKGROUND

[0002] New intent discovery is an important task of intelligent customer service question answering system, which can not only improve the service quality of the question answering system, but also find potential user interests that may provide business opportunities and guide improvement direction by analyzing the discovered new intent. New intent discovery is usually regarded as a clustering problem.

[0003] In actual application scenarios, a large number of questions are unlabeled annotated data, and only a small amount of questions are labeled annotated data. How to fully utilize the unlabeled annotated data and a small amount of labeled annotated data for clustering analysis is a problem to be solved by those skilled in the art. SUMMARY

[0004] The present application provides a model training and intelligent customer service question clustering method and related device, which can fully utilize labeled annotated data and unlabeled annotated data when training a clustering feature representation model, so that the clustering effect of the trained clustering feature representation model is better, and finally the clustering feature representation model can accurately cluster intelligent customer service questions according to the clustering result and timely discover new user intent.

[0005] In order to achieve the above purpose, the technical scheme adopted by the embodiments of the present application is as follows:

[0006] In a first aspect, the embodiments of the present application provide a model training method, which comprises:

[0007] inputting a training set into a clustering feature representation model to obtain a feature vector of each training data, wherein the training set comprises labeled training data and unlabeled training data;

[0008] clustering each feature vector to obtain a clustering cluster and a cluster label corresponding to each clustering cluster;

[0009] when the clustering cluster does not satisfy a clustering condition, iteratively optimizing each cluster label according to a label set and / or each cluster label, wherein the label set comprises a true label of each labeled training data in the training set;

[0010] training the clustering feature representation model based on the training data and the optimized cluster label to obtain a trained clustering feature representation model.

[0011] Optionally, the step of iteratively optimizing each of the cluster labels according to the label set and / or each of the cluster labels comprises:

[0012] According to the true labels of the labeled training data, modifying the cluster labels corresponding to the labeled training data to obtain the optimized cluster labels.

[0013] Optionally, the step of iteratively optimizing each of the cluster labels according to the label set and / or each of the cluster labels further comprises:

[0014] aligning the cluster labels of the current iteration with the cluster labels of the previous iteration to obtain the optimized cluster labels, wherein the cluster labels of the labeled training data in the cluster labels of the current iteration are specifically the optimized cluster labels obtained by modifying the cluster labels corresponding to the labeled training data according to the true labels of the labeled training data.

[0015] Optionally, the step of training the clustering feature representation model based on the training data and the optimized cluster labels to obtain a trained clustering feature representation model comprises:

[0016] performing parameter optimization on the clustering feature representation model with the training data, the cluster labels corresponding to the unlabeled training data, and the optimized cluster labels to obtain an optimized clustering feature representation model;

[0017] inputting the training set into the optimized clustering feature representation model to obtain an optimized feature vector of each of the training data;

[0018] performing clustering on each of the optimized feature vectors to obtain optimized clustering clusters and an optimized cluster label corresponding to each of the optimized clustering clusters;

[0019] when the optimized clustering clusters satisfy a clustering condition, the optimized clustering feature representation model is the trained clustering feature representation model.

[0020] Optionally, the method further comprises:

[0021] for each feature vector, calculating an intra-cluster distance of the feature vector and an inter-cluster distance of the feature vector, the intra-cluster distance being an average distance of the feature vector to other feature vectors in a target clustering cluster to which the feature vector belongs, and the inter-cluster distance being a minimum value of average distances of the feature vector to all feature vectors in each clustering cluster other than the target clustering cluster;

[0022] calculating a silhouette coefficient according to the intra-cluster distance and the inter-cluster distance of each feature vector;

[0023] if the silhouette coefficient converges, determining that the clustering clusters satisfy a clustering condition.

[0024] If the profile coefficient does not converge, it is determined that the clustering cluster does not satisfy the clustering condition.

[0025] Optionally, the step of modifying the cluster label corresponding to the labeled training data according to the true label of the labeled training data to obtain the optimized cluster label comprises:

[0026] obtaining a category center of each true label according to a feature vector corresponding to each true label;

[0027] obtaining a cluster center of each cluster label according to a feature vector corresponding to each cluster label;

[0028] determining a first mapping relationship according to each category center and each cluster center, the first mapping relationship representing a one-to-one mapping relationship between all true labels and all cluster labels;

[0029] modifying the cluster label of each labeled training data to the cluster label corresponding to the true label of each labeled training data according to the first mapping relationship.

[0030] Optionally, the cluster label of the previous iteration is taken as a reference label, and the cluster label of the current iteration is taken as a label to be aligned, and the step of aligning the cluster label of the current iteration with the cluster label of the previous iteration to obtain the optimized cluster label comprises:

[0031] obtaining a reference cluster center of each reference label according to a feature vector corresponding to each reference label;

[0032] obtaining a to-be-aligned cluster center of each label to be aligned according to a feature vector of the training data corresponding to each label to be aligned;

[0033] determining a second mapping relationship according to each reference cluster center and each to-be-aligned cluster center, the second mapping relationship representing a one-to-one mapping relationship between all reference labels and all labels to be aligned;

[0034] updating the label to be aligned of the training data to the reference label corresponding to the label to be aligned according to the second mapping relationship to obtain the optimized cluster label of the training data.

[0035] In a second aspect, an embodiment of the present application provides an intelligent customer service question clustering method, and the method comprises:

[0036] obtaining intelligent customer service questions to be clustered;

[0037] The model trained by the model training method in the first aspect is used to cluster the intelligent customer service question, and a clustering result of the intelligent customer service question is obtained.

[0038] In a third aspect, an embodiment of the present application provides a model training device, and the device comprises:

[0039] a vector generation module configured to input a training set into a clustering feature representation model to obtain a feature vector of each training data, the training set comprising labeled training data and unlabeled training data;

[0040] a clustering module configured to cluster each feature vector to obtain a clustering cluster and a cluster label corresponding to each clustering cluster;

[0041] a self-supervised learning module configured to, when the clustering cluster does not satisfy a clustering condition, iteratively optimize each cluster label according to a label set and / or each cluster label, the label set comprising a true label of each labeled training data in the training set; and train the clustering feature representation model based on the training data and the optimized cluster label to obtain a trained clustering feature representation model.

[0042] In a fourth aspect, an embodiment of the present application provides an intelligent customer service question clustering device, and the device comprises:

[0043] a question acquisition module configured to acquire an intelligent customer service question to be clustered;

[0044] a clustering result generation module configured to use a model trained by the model training method in the first aspect to cluster the intelligent customer service question, and obtain a clustering result of the intelligent customer service question.

[0045] In a fifth aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, the memory being configured to store a program, and the processor being configured to, when executing the program, implement the model training method in the first aspect and / or the intelligent customer service question clustering method in the second aspect.

[0046] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, and the computer program is configured to, when executed by a processor, implement the model training method in the first aspect and / or the intelligent customer service question clustering method in the second aspect.

[0047] Compared with the prior art, the embodiment of the present application provides a model training, intelligent customer service question clustering method and related device, a training set and a label set are obtained, the training set includes labeled training data and unlabeled training data, the label set includes a real label of each labeled training data in the training set, the training set is input into a clustering feature representation model to obtain a feature vector of each training data, each feature vector is clustered to obtain a clustering cluster and a cluster label corresponding to each clustering cluster, when the clustering cluster does not satisfy a clustering condition, each cluster label is iteratively optimized according to the label set and / or each cluster label, the label set includes the real label of each labeled training data in the training set; based on the training data and the optimized cluster label, the clustering feature representation model is trained to obtain a trained clustering feature representation model, in the embodiment, the labeled data and the unlabeled data are fused to train the clustering feature representation model, the labeled data and the unlabeled data can be fully utilized, the clustering effect of the trained clustering feature representation model is better, and finally the intelligent customer service question can be accurately clustered by using the clustering feature representation model, and new user intentions can be discovered in time according to the clustering result. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0049] Figure 1 Flowchart of a model training method provided by the embodiment of the present application Figure 1 .

[0050] Figure 2 Flowchart of a model training method provided by the embodiment of the present application Figure 2 .

[0051] Figure 3 Example diagram of intra-cluster distance provided by the embodiment of the present application.

[0052] Figure 4 Example diagram of inter-cluster distance provided by the embodiment of the present application.

[0053] Figure 5 Example diagram of a correction process provided by the embodiment of the present application.

[0054] Figure 6 Example diagram of an alignment process provided by the embodiment of the present application.

[0055] Figure 7A flowchart of an intelligent customer service question clustering method provided for an embodiment of the present application.

[0056] Figure 8 An example diagram of partition setting for any data set is provided for the present embodiment.

[0057] Figure 9 A block diagram of a model training device provided for an embodiment of the present application.

[0058] Figure 10 A block diagram of an intelligent customer service question clustering device provided for an embodiment of the present application.

[0059] Figure 11 A block diagram of an electronic device provided for an embodiment of the present application.

[0060] Icon: 100 - model training device; 110 - vector generation module; 120 - clustering module; 130 - self-supervised learning module; 200 - intelligent customer service question clustering device; 210 - question acquisition module; 220 - clustering result generation module; 10 - electronic device; 11 - processor; 12 - memory; 13 - bus. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0063] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0064] In the description of the present application, it should be noted that if the terms "upper", "lower", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, or the orientation or positional relationship commonly used when the product of the present application is used, only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0065] In addition, if the terms "first", "second" and the like appear, they are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0066] It should be noted that the features in the embodiments of the present application can be combined with each other without conflict.

[0067] Since cluster analysis can mine the internal structure of data, data with similar attributes are clustered into a class, so that the data in the same class has high similarity, therefore, in the intelligent customer service question and answer system, how to cluster the questions in the intelligent customer service question and answer system helps to detect new user intent, and a currently used clustering method is usually an unsupervised clustering method, in which the sample labels of sample data used by the method are unknown, the sample data set is divided into different class clusters according to a certain standard (for example, distance), and finally the sample data in the same class cluster is as similar as possible, and the sample data in different class clusters is as dissimilar as possible. The method is limited to using only a large amount of unlabeled training data, without using a small amount of labeled training data to guide the clustering process, so the efficiency of the clustering process is not high, and the clustering result often cannot meet the expectation.

[0068] In view of the defects of the unsupervised clustering method, there is also a weakly supervised clustering method, in which the pre-trained model is fine-tuned using labeled data, and then the feature vector representation of unlabeled data is obtained based on the fine-tuned pre-trained model, and the feature vectors of the unlabeled data are clustered and mined. Although this method can introduce a small amount of labeled prior knowledge to guide the clustering process and obtain knowledge from labeled data, it improves the efficiency of the clustering process and improves the clustering effect to some extent. However, although the method uses labeled data to fine-tune the pre-trained model, it only uses unlabeled data in the clustering process, so it does not fully integrate labeled data and unlabeled data, and fully mine the internal relationship knowledge of the data, so the improvement of the clustering efficiency and the clustering effect is extremely limited.

[0069] Therefore, the embodiments of the present application provide a model training, intelligent customer service question clustering method and related device, and the core scheme mainly includes two stages: knowledge transfer and weakly supervised deep alignment clustering.

[0070] In the knowledge transfer stage, in order to fully utilize a small amount of labeled training data for knowledge transfer, on the one hand, the labeled training data and the real labels thereof are used to fine-tune the pre-constructed clustering feature representation model to obtain a fine-tuned clustering feature representation model, and on the other hand, as a relatively optimal implementation manner, the fine-tuned clustering feature representation model is used for clustering class number prediction to determine the clustering class number required by the weakly supervised deep alignment clustering stage, and of course, the clustering class number can also be set as an empirical value.

[0071] In the weakly supervised deep alignment clustering stage, the determined clustering class number is used for K-means clustering of the feature vectors of the training data to obtain clustering clusters of the clustering class number and initial cluster labels of the training data corresponding to each clustering cluster, in the training process, the initial cluster labels are adjusted according to the real labels of the labeled training data, and then the training data and the adjusted cluster labels are used for self-supervised learning of the clustering feature representation model, and finally the trained clustering feature representation model is obtained. The present application constructs a high-quality self-supervised signal by introducing the labeled training data to correct the cluster labels obtained in the clustering process, guides the clustering learning process, fully utilizes the clustering knowledge mining of the fusion of the labeled training data and the unlabeled training data, and can greatly improve the clustering efficiency and obviously improve the clustering effect, which will be described in detail below.

[0072] It should be noted that the above two stages are independent of each other, and can be processed on different electronic devices or on the same electronic device.

[0073] Please refer to Figure 1 , Figure 1 The flowchart of a model training method provided by the embodiment of the present application Figure 1 The method comprises the following steps:

[0074] Step S100, inputting a training set into a clustering feature representation model to obtain a feature vector of each training data, the training set comprising labeled training data and unlabeled training data.

[0075] In the embodiment, for the application scene of intelligent customer service question clustering, the training data is a question in text form, which can be directly collected from an intelligent customer service system as training data, or a voice question collected from an intelligent customer service system can be converted into a question in text form as training data.

[0076] In the embodiment, the true label of the training data can be the label of the user intent category represented by the training data, for example, the training set includes four training data, respectively: "Hmm, the fund transfer cannot be transferred out.", "Hmm, the transfer cannot be transferred.", "Hmm, the fund cannot be transferred out.", "The account cannot be traded.", the true labels of the first three training data are "transfer failure", and the label of the last training data is "non-tradable".

[0077] In the embodiment, the clustering feature representation model can be a pre-constructed bert model, which is fine-tuned in the knowledge transfer stage described above. The fine-tuning process can be: first, represent the training data as (X i , Y i ), where X i represents the text data of the i-th question, Y i represents the true label of the i-th question, then use the preset cross-entropy loss function, use the labeled training data and its true label to fine-tune the clustering feature representation model by taking the intent classification as a downstream task, and obtain the fine-tuned clustering feature representation model.

[0078] In the embodiment, the feature vector is a vector representation of the training data, and each training data corresponds to a feature vector. When the structure of the clustering feature representation model is unchanged, the feature vectors obtained by inputting the same training data into different parameter clustering feature representation models can be different.

[0079] Step S101, clustering each feature vector to obtain a clustering cluster and a cluster label corresponding to each clustering cluster.

[0080] In the embodiment, the number of clustering clusters, i.e. the number of clustering categories in the knowledge transfer stage described above, can be determined according to an empirical value in advance, or can be predicted in the knowledge transfer stage described above by clustering. The specific prediction method can be: first, define an approximate initial cluster number K' according to the number of known intents (for example, the value of K' is defined as 1.5 times the number of known intents), then input the training data (including labeled training data and unlabeled training data) into the clustering feature representation model to obtain the feature vectors of the training data, next, perform K-means clustering on the feature vectors of the training data to obtain K' clustering clusters, and finally, predict the set number K according to the K' clustering clusters.

[0081] Since the size of different clusters is different when the number of clusters is large, the more training data a cluster contains, the more accurate the division of the cluster is, and thus the number of training data contained in the cluster is used as its confidence. The cluster feature representation model is fine-tuned only based on the labeled training data, but the weak semantic similarity features learned by the cluster feature representation model are still more beneficial to new intent clustering. Therefore, by removing the clusters with low confidence, the number of clusters with confidence higher than a preset threshold is calculated to predict K, and the calculation formula of K is as follows:

[0082]

[0083] wherein K' is the initial number of clusters, K is the set number, t represents the preset threshold, |S i is the confidence of cluster i, i.e., the number of training data included in cluster i, and δ represents the number of clusters with confidence greater than or equal to t.

[0084] As a specific embodiment, t can be set as the average confidence, and the calculation method can be: when the training data (including labeled training data and unlabeled training data) is divided into K' clusters, the average number of training data in each cluster.

[0085] In the embodiment, the cluster labels of training data in the same cluster are the same. A cluster can include labeled training data, unlabeled training data, or both labeled training data and unlabeled training data.

[0086] For fine-tuning of the cluster feature representation model and determination of the set number, refer to the foregoing, which is used to fine-tune the cluster feature representation model by using the labeled training data in the "knowledge transfer stage", and determine the set number by using the labeled training data and the unlabeled training data, so as to fully utilize the labeled training data for knowledge transfer, so that the clustering effect of the finally trained cluster feature representation model is better.

[0087] In step S102, when the cluster does not meet the clustering condition, each cluster label is iteratively optimized according to the label set and / or each cluster label. The label set includes the real label of each labeled training data in the training set.

[0088] In the embodiment, the clustering can be performed once, and the obtained clustering cluster satisfies the clustering condition, or the clustering can be performed multiple times, and the clustering cluster obtained finally satisfies the clustering condition. When the clustering is performed multiple times, the cluster label needs to be iteratively optimized. The purpose of the iterative optimization is to make the cluster label of each training data after the optimization more reasonable. The iterative optimization of the cluster label can be modifying the cluster label after the clustering according to the real label, or aligning the cluster label of the current iteration with the cluster label of the previous iteration.

[0089] In the embodiment, the clustering condition is used to represent whether the clustering result of the clustering cluster converges or whether the clustering times satisfy the clustering condition. Whether the clustering result converges can be determined by whether the clustering result of the feature vector in each clustering cluster in a continuous preset number of times satisfies a preset similarity. For example, the number of clustering clusters is 3, and the clustering results of the continuous 3 times of clustering are {a}, {b, c}, and {d, e}. It is determined that the clustering cluster satisfies the clustering condition. The clustering result can also be determined by whether the clustering times satisfy the clustering condition. For example, if the clustering times reach 10 times, it is determined that the clustering condition is satisfied.

[0090] In step S103, the clustering feature representation model is trained based on the training data and the optimized cluster label, and a trained clustering feature representation model is obtained.

[0091] In the embodiment, when the clustering feature representation model is trained, the label of the training data used is the cluster label obtained by optimizing the initial cluster label according to the label set. Therefore, the labeled training data and the unlabeled training data can be used to train the clustering feature representation model, the labeled training data can be used to guide the clustering learning process, and the labeled annotation data and the unlabeled annotation data can be fully utilized.

[0092] For the training process of the clustering feature representation model in steps S100-S103, refer to the foregoing description. The training process is used to perform the clustering feature representation model training by fusing the labeled training data and the unlabeled training data in the weakly supervised deep alignment clustering stage, to guide the clustering learning process by using the labeled training data, to fully utilize the labeled annotation data and the unlabeled annotation data, and to make the clustering effect of the trained clustering feature representation model better.

[0093] In Figure 1On the basis of the above, in order to make the training process of the clustering feature representation model be able to terminate the training in time when the demand is met, and avoid repeated invalid training, the embodiment further adds the judgment of whether the clustering condition is met, and the clustering cluster meeting the clustering condition means that the clustering process has met the convergence condition, that is, the clustering result of this training and the clustering result of the last training are different within a preset range, or the difference of the clustering results of continuous several times of training are all within a preset range, at this time, continuing training and clustering cannot obtain better clustering result, and thus the training can be stopped.

[0094] As a specific implementation, the process of judging whether the clustering cluster meets the clustering condition can be:

[0095] Firstly, for each feature vector, the intra-cluster distance and the inter-cluster distance of the feature vector are calculated, the intra-cluster distance is the average distance of the feature vector and other feature vectors in the target clustering cluster to which the feature vector belongs, and the inter-cluster distance is the minimum value in the average distance of the feature vector and all feature vectors in each other clustering cluster except the target clustering cluster.

[0096] Specifically, in the embodiment, the intra-cluster distance can be used to quantify the cohesion degree in the cluster, and the inter-cluster distance is used to quantify the separation degree between clusters, the intra-cluster distance of the feature vector represents the intra-cluster dissimilarity of the feature vector, and the smaller the intra-cluster distance is, the more the feature vector should be clustered to the clustering cluster. Please refer to Figure 3 , Figure 3 The example diagram of the intra-cluster distance provided by the embodiment of the application is shown in the following figure. Figure 3 In the figure, cluster a includes three feature vectors: vector 1~vector 3, taking vector 1 as an example, the distance between vector 1 and vector 2 is d1, and the distance between vector 1 and vector 3 is d2, and then the intra-cluster distance of vector 1 is the average of d1 and d2. Please refer to Figure 4 , Figure 4 The example diagram of the inter-cluster distance provided by the embodiment of the application is shown in the following figure, Figure 4 In the figure, taking vector 12 as an example, vector 12 belongs to cluster a, the distance between vector 12 and vector 21 in cluster b is d11, and the distance between vector 12 and vector 22 in cluster b is d12, the distance between vector 12 and vector 31 in cluster c is d21, the distance between vector 12 and vector 32 in cluster c is d22, and the distance between vector 12 and vector 33 in cluster c is d23, and then the inter-cluster distance is min(avg(d11, d12), avg(d21, d22, d23)), wherein avg represents an average value function, and min represents a minimum value function.

[0097] Secondly, according to the intra-cluster distance and the inter-cluster distance of each feature vector, the silhouette coefficient is calculated, if the silhouette coefficient converges, it is determined that all clustering clusters meet the preset condition, and if the silhouette coefficient does not converge, it is determined that all clustering clusters do not meet the preset stop condition.

[0098] In this embodiment, the larger the silhouette coefficient value, the more reasonable the clustering result. The silhouette coefficient can be calculated using the following formula:

[0099]

[0100] Where N represents the number of training data, SC represents the silhouette coefficient, a(I i ) represents the intra-cluster distance of feature vector i, b(I i ) represents the inter-cluster distance of feature vector i, max is the maximum value function, and the specific calculation method of the inter-cluster distance is:

[0101] b(I i )=min{b i1 ,b i2 ,...,b ik}

[0102] Among them, k is the number of clusters, b ij It represents the average distance from feature vector i to all feature vectors in cluster j, and min is the minimum value function.

[0103] exist Figure 1 Based on this, the clustering feature representation model can be obtained after iterative training. This embodiment also provides a specific implementation method for training the clustering feature representation model. Please refer to Figure 2 , Figure 2 A schematic diagram of a model training method provided in an embodiment of the present invention Figure 2 , step S103 includes the following sub-steps:

[0104] Sub-step S1031 , performing parameter optimization on the cluster feature representation model using the training data, the cluster labels corresponding to the unlabeled training data, and the optimized cluster labels to obtain the optimized cluster feature representation model.

[0105] In this embodiment, the cluster labels corresponding to the unlabeled training data can be obtained by the first clustering, or obtained after multiple iterative optimizations, and each iterative alignment aligns the cluster labels of the current iteration with the cluster labels of the previous iteration. The cluster labels of the labeled training data can be obtained by modifying the cluster labels obtained by the first clustering according to the true labels, or obtained after multiple iterative optimizations, and each iterative optimization first modifies the cluster labels of the current iteration according to the true labels, and then aligns the modified cluster labels with the cluster labels of the previous iteration.

[0106] In this embodiment, the training data, the cluster labels corresponding to the unlabeled training data, and the optimized cluster labels are input into the current optimized cluster feature representation model, and the parameters of the cluster feature representation model are optimized in combination with the preset cross entropy loss function to obtain the optimized cluster feature representation model.

[0107] Sub-step S1032, inputting the training set into the optimized clustering feature representation model to obtain an optimized feature vector of each training data.

[0108] Sub-step S1033, clustering each optimized feature vector to obtain an optimized clustering cluster and an optimized cluster label corresponding to each optimized clustering cluster.

[0109] In this embodiment, the processing procedure of sub-step S1033 and step S101 is similar, and will not be described here.

[0110] Sub-step S1034, when the optimized clustering cluster satisfies the clustering condition, the optimized clustering feature representation model is a trained clustering feature representation model.

[0111] In this embodiment, when the optimized clustering cluster does not satisfy the clustering condition, the processing procedure is similar to that of step S102, except that the optimized clustering cluster in sub-step S1034 is used to replace the clustering cluster in step S102, the optimized cluster label in sub-step S1034 is used to replace the cluster label in step S102, and the remaining procedure is similar to that of step S102, which will not be described here.

[0112] In this embodiment, for the labeled training data, the process of optimizing the cluster label at least exists in two cases:

[0113] Case one: when the current optimization is the first iteration of optimization, only the cluster label of the labeled training data needs to be optimized according to the real label, specifically:

[0114] According to the real label of the labeled training data, the cluster label corresponding to the labeled training data is modified to obtain an optimized cluster label.

[0115] Case two: when the current optimization is not the first iteration of optimization, first, the cluster label of the labeled training data is modified according to the real label, and then, the cluster label modified in the current iteration and the cluster label in the previous iteration are aligned, specifically:

[0116] Aligning the cluster label in the current iteration with the cluster label in the previous iteration to obtain an optimized cluster label, wherein the cluster label of the labeled training data in the cluster label in the current iteration is specifically: the optimized cluster label obtained by modifying the cluster label corresponding to the labeled training data according to the real label of the labeled training data.

[0117] In this embodiment, for the unlabeled training data, the process of optimizing the cluster label at least exists in two cases:

[0118] Case one: when the current optimization is the first iteration of optimization, the unlabeled training data does not need to be optimized.

[0119] Case two: when the current iteration modified cluster label and the cluster label of the previous iteration are aligned in the same way as the labeled training data, the optimization is performed in the current iteration of the optimization.

[0120] In this embodiment, since the true label of the labeled training data can represent the correct type it belongs to, and the initial cluster label of the labeled training data determined by clustering may not be consistent with the true label it belongs to, in order to make the initial cluster label of the labeled training data more accurate, the embodiment also provides a specific implementation manner of modifying the cluster label corresponding to the labeled training data according to the true label, the process is as follows:

[0121] (1) obtaining a class center of each true label according to the feature vector corresponding to each true label;

[0122] As a specific implementation manner, the average value of the feature vector corresponding to each true label can be calculated as the class center of each true label.

[0123] (2) obtaining a cluster center of each cluster label according to the feature vector corresponding to each cluster label;

[0124] As a specific implementation manner, the average value of the feature vector corresponding to each cluster label can be calculated as the cluster center of each cluster label.

[0125] (3) determining a first mapping relationship according to each class center and each cluster center, the first mapping relationship representing a one-to-one mapping relationship between all true labels and all cluster labels;

[0126] In this embodiment, each true label has a class center, and each cluster label has a cluster center, based on the Euclidean distance between each class center and each cluster center, the Hungarian algorithm is used to obtain the first mapping relationship in the Euclidean space, the first mapping relationship can be represented as: C1 c = M(L r ), wherein C1 c represents the cluster center, L r represents the class center, and M represents the first mapping relationship.

[0127] (4) modifying the cluster label of each labeled training data to the cluster label corresponding to the true label of each labeled training data according to the first mapping relationship.

[0128] The formula for modifying the labeled training data according to the first mapping relationship can be represented as:

[0129] wherein, denotes the modified cluster label, M -1 denotes the inverse mapping of M, y r denotes the cluster label.

[0130] Please refer to Figure 5 , Figure 5 the example diagram of the modification process provided by the inventive embodiment, Figure 5 In the example, the labeled training data are S1-S5, wherein the true labels of S1-S3 are T1, and the true labels of S4 and S5 are T2. The unlabeled training data are S6-S9. First, the feature vectors of S1-S9 are clustered to obtain three cluster labels, C1, C2 and C3, and the corresponding training data are {S1, S2, S6}, {S4, S5, S3} and {S7, S8, S9} respectively. Then, the first mapping relationship C1-T1 and C2-T2 is obtained in the manner described above. Finally, the modification is performed. After the modification, the corresponding training data of C1, C2 and C3 are {S1, S2, S3, S6}, {S4, S5} and {S7, S8, S9} respectively.

[0131] In the embodiment, the alignment process of the labeled training data is the same as that of the unlabeled training data. The only difference is that the cluster label of the labeled training data to be aligned in the current iteration is the cluster label modified according to the true label, and the cluster label of the unlabeled training data to be aligned in the current iteration is the cluster label obtained by clustering in the current iteration. Therefore, in the description of the alignment process, the labeled training data and the unlabeled training data are not distinguished. In order to facilitate the description, the cluster label of the previous iteration is taken as the reference label, and the cluster label of the current iteration is taken as the label to be aligned. The specific alignment process is as follows:

[0132] (1) Obtain the reference cluster center of each reference label according to the feature vector corresponding to each reference label.

[0133] (2) Obtain the to-be-aligned cluster center of each label to be aligned according to the feature vector of the training data corresponding to each label to be aligned.

[0134] In the embodiment, the determination manner of the reference cluster center and the determination manner of the to-be-aligned cluster center are the same as the determination manner of the cluster center in the modification process, which will not be described herein.

[0135] (3) Determine the second mapping relationship according to each reference cluster center and each to-be-aligned cluster center, wherein the second mapping relationship represents a one-to-one mapping relationship between all reference labels and all labels to be aligned.

[0136] In the embodiment, the determination manner of the second mapping relationship is the same as that of the first mapping relationship. The second mapping relationship can be represented as:

[0137] C2 c =G(C2 l ), wherein C2 c represents a to-be-aligned cluster center, C2 l represents a reference cluster center, and G represents a second mapping relationship.

[0138] (4) According to the second mapping relationship, updating the to-be-aligned label of the training data to a reference label corresponding to the to-be-aligned label to obtain an optimized cluster label of the training data.

[0139] The formula for aligning to obtain the cluster label according to the second mapping relationship can be represented as:

[0140] wherein y align represents a cluster label, represents a to-be-aligned label, G -1 represents an inverse mapping of G.

[0141] Please refer to Figure 6 , Figure 6 an example diagram of an alignment process provided by the embodiment of the application, Figure 6 wherein the to-be-aligned labels are C1, C2 and C3, the corresponding training data of the to-be-aligned labels are {S1, S2, S3, S6}, {S4, S5} and {S7, S8, S9} respectively, the reference labels are Ca, Cb and Cc, the corresponding training data of the reference labels are {S1, S2, S3, S7}, {S4, S5} and {S6, S8, S9} respectively, the second mapping relationship is C1-Ca, C2-Cb and C3-Cc, and the cluster labels after alignment are Ca, Cb and Cc, and the corresponding training data of the cluster labels are {S1, S2, S3, S6}, {S4, S5} and {S7, S8, S9} respectively.

[0142] In the embodiment, in order to correctly cluster intelligent customer service questions, so as to discover new user intents in time, the embodiment further provides an intelligent customer service question clustering method, please refer to Figure 7 , Figure 7 a flowchart of an intelligent customer service question clustering method provided by the embodiment of the application, the method comprising the following steps:

[0143] Step S200, obtaining intelligent customer service questions to be clustered.

[0144] Step S201, clustering the intelligent customer service questions by using a model obtained by the model training method provided by the embodiment to obtain a clustering result of the intelligent customer service questions.

[0145] The method provided in the embodiment can realize accurate clustering, and further discover new user intents in a timely and accurate manner according to the accurate clustering results.

[0146] To prove the effectiveness of the clustering feature representation model trained in the embodiment, the deep alignment clustering, the hierarchical clustering and the iterative self-correction clustering provided in the embodiment (that is, clustering is performed by using the model obtained by the model training method provided in the embodiment) are respectively used to cluster different data sets, and the clustering results are compared. The experimental results show that the iterative self-correction clustering provided in the embodiment has better clustering effect than the other two clustering methods. The specific experiment is as follows:

[0147] 1) Data set:

[0148] The following three types of data sets are sampled for training in the embodiment:

[0149] ①stock data set: Chinese data set, securities field data set, total amount of training set data 3722, total number of categories 44, and the amount of labeled data in the training set is 258.

[0150] ②banking data set: English data set, bank field fine-tuning data set, total amount of training set data 9016, total number of categories 77, and the amount of labeled data in the training set is 673.

[0151] ③clinc data set: English data set, intent classification data set, total amount of training set data 18000, total number of categories 150, and the amount of labeled data in the training set is 1344.

[0152] 2) Data set division setting:

[0153] For each data set, the same division method is used. For any data set, please refer to Figure 8 , Figure 8 The example of the division setting of any data set provided in the embodiment is shown in Figure 8 , the data set is divided into a training set, a validation set and a test set according to the proportions of 80%, 10% and 10%, the training set is used to train the clustering feature representation model, the validation set is used to verify the clustering effect of the trained clustering feature representation model, and the test set is used to test the trained clustering feature representation model. Among them, the proportions of labeled data and unlabeled data in the training set are 75% and 25% respectively, for the labeled data, 10% of the labeled data is used as labeled training data, 90% of the labeled data is used as unlabeled training data, and all the unlabeled data is used as unlabeled training data.

[0154] 3) Evaluation index:

[0155] ① ACC: Accuracy, the cluster label class is mapped to the real label class by using the Hungarian algorithm, and then the accuracy is calculated, which is an evaluation of the accuracy of cluster division, the value range is [0, 1], the larger the value, the better the clustering result.

[0156] ② ARI: Adjusted Rand Index, the value range is [-1, 1], which is used to measure the consistency of the clustering result and the real situation, the larger the value, the more consistent, which can be used for effect comparison between clustering algorithms.

[0157] ③ NMI: Normalized Mutual Information, which is used to measure the consistency of two independent label assignment strategies on the same data set, the value range is [0, 1], the closer the value to 1, the better the clustering effect.

[0158] 4) Evaluation result analysis:

[0159] Iterative self-correction clustering is better than deep alignment clustering and hierarchical clustering on three data sets and three evaluation indexes. Compared with deep alignment clustering, the accuracy is improved by 0.9-1.6%, and compared with hierarchical clustering, the accuracy is improved by 35-62%, which shows that iterative self-correction clustering can more effectively use labeled training data to construct high-quality supervised signals and fully utilize the fusion of labeled training data and unlabeled training data for knowledge mining to get better clustering effect. The model evaluation results of the three data sets are shown in Table 1.

[0160] Table 1

[0161]

[0162] 5) Clustering result example:

[0163] The specific clustering results of the three clustering methods are shown in Table 2:

[0164] Table 2

[0165]

[0166]

[0167] In Table 2,

[0168] Field "question": the question used for clustering

[0169] Field "standard question": the category to which the clustering question belongs

[0170] The field contains "cluster label": the clustering result of the corresponding method, where each number represents the cluster result label, and the same number indicates that it belongs to the same cluster category.

[0171] From the clustering results of Table 2, we can get:

[0172] For the "Beijing Stock Exchange new stock how to subscribe" data, the iterative self-correction clustering result accurately clusters this type of data into one class, the deep alignment clustering divides it into two categories, and the hierarchical clustering divides it into three categories. The hierarchical clustering result deviates from the actual result more.

[0173] For the "transfer failed" data; iterative self-correction clustering clusters all data belonging to this category into one class, while deep alignment clustering clusters the question "Hmm, the funds cannot be transferred out." which is obviously related to "transfer failed" into another category, and the hierarchical clustering result is more fragmented, directly dividing the three data under this category into three categories. The hierarchical clustering result deviates from the actual result more.

[0174] For the "stock cannot be traded" data, iterative self-correction clusters all data belonging to this category into one class, while deep alignment clustering divides the question "Shanghai and Shenzhen cannot be bought." which is obviously related to "stock cannot be traded" into another category, which does not match the actual annotation, while hierarchical clustering divides each question under this category into a separate class, which deviates from the actual result.

[0175] For the "operational risk assessment" data, the iterative self-correction clustering result accurately clusters this type of data into one class, the deep alignment clustering divides it into two categories, and the hierarchical clustering divides it into three categories. The hierarchical clustering result deviates from the actual result more.

[0176] Deep alignment clustering does not cluster questions that obviously belong to the same category into one category, mainly because the cluster label is not corrected during the clustering process based on labeled training data, and the clustering feature representation model does not learn the correct classification knowledge. Iterative self-correction clustering fully integrates labeled training data and unlabeled training data, and uses labeled training data to correct incorrect cluster labels during the clustering process, guiding the model to learn correctly, thereby obtaining a result that is more consistent with the actual result. Based on this result, the iterative self-correction clustering result is better than the deep alignment clustering result, which is better than the hierarchical clustering result.

[0177] To perform the corresponding steps in the above embodiments and various possible implementations, the following respectively gives an implementation of a model training apparatus 100 and an intelligent customer service question clustering apparatus 200, which are applied to the same or different electronic devices. It should be noted that the model training apparatus 100 and the intelligent customer service question clustering apparatus 200 provided in this embodiment have the same basic principles and technical effects as the corresponding above-mentioned embodiments, and for brief description, part of this embodiment is not mentioned.

[0178] Please refer to Figure 9 , Figure 9 The block schematic diagram of the model training apparatus 100 provided in the embodiments of the present application is shown.

[0179] The model training apparatus 100 comprises a vector generation module 110, a clustering module 120 and a self-supervised learning module 130.

[0180] The vector generation module 110 is configured to input a training set into a clustering feature representation model to obtain a feature vector of each training data, wherein the training set comprises labeled training data and unlabeled training data.

[0181] The clustering module 120 is configured to cluster each feature vector to obtain a clustering cluster and a cluster label corresponding to each clustering cluster.

[0182] The self-supervised learning module 130 is configured to, when the clustering cluster does not satisfy a clustering condition, iteratively optimize each cluster label according to a label set and / or each cluster label, wherein the label set comprises a true label of each labeled training data in the training set; and train the clustering feature representation model based on the training data and the optimized cluster label to obtain a trained clustering feature representation model.

[0183] The model training apparatus 100 provided in this embodiment performs clustering feature representation model training by fusing labeled training annotation data and unlabeled training annotation data, uses labeled training data to guide the clustering learning process, achieves the purpose of making full use of labeled annotation data and unlabeled annotation data, and makes the clustering effect of the trained clustering feature representation model better.

[0184] Optionally, the self-supervised learning module 130 is specifically configured to modify the cluster label corresponding to the labeled training data according to the true label of the labeled training data to obtain the optimized cluster label.

[0185] Optionally, the self-supervised learning module 130 is specifically further configured to align the cluster label of the current iteration with the cluster label of the previous iteration to obtain the optimized cluster label, wherein the cluster label of the labeled training data in the cluster label of the current iteration is specifically the optimized cluster label obtained by modifying the cluster label corresponding to the labeled training data according to the true label of the labeled training data.

[0186] Optionally, the self-supervised learning module 130 is specifically configured to: perform parameter optimization on the clustering feature representation model according to the training data, the cluster labels corresponding to the unlabeled training data, and the optimized cluster labels, to obtain an optimized clustering feature representation model; input the training set into the optimized clustering feature representation model to obtain an optimized feature vector of each training data; perform clustering on each optimized feature vector to obtain optimized clustering clusters and an optimized cluster label corresponding to each optimized clustering cluster; and when the optimized clustering clusters satisfy a clustering condition, the optimized clustering feature representation model is a trained clustering feature representation model.

[0187] Optionally, the self-supervised learning module 130 is further configured to: for each feature vector, calculate an intra-cluster distance and an inter-cluster distance of the feature vector, the intra-cluster distance being an average distance between the feature vector and other feature vectors in a target clustering cluster to which the feature vector belongs, and the inter-cluster distance being a minimum value of an average distance between the feature vector and all feature vectors in each other clustering cluster except the target clustering cluster; calculate a silhouette coefficient according to the intra-cluster distance and the inter-cluster distance of each feature vector; and determine that the clustering clusters satisfy the clustering condition if the silhouette coefficient converges, or determine that the clustering clusters do not satisfy the clustering condition if the silhouette coefficient does not converge.

[0188] Optionally, when the self-supervised learning module 130 is configured to modify the cluster labels corresponding to the labeled training data according to the real labels of the labeled training data to obtain the optimized cluster labels, the self-supervised learning module 130 is specifically configured to: obtain a class center of each real label according to a feature vector corresponding to the real label; obtain a cluster center of each cluster label according to a feature vector corresponding to the cluster label; determine a first mapping relationship according to each class center and each cluster center, the first mapping relationship representing a one-to-one mapping relationship between all real labels and all cluster labels; and modify, according to the first mapping relationship, the cluster label of each labeled training data to a cluster label corresponding to a real label of each labeled training data.

[0189] Optionally, when the self-supervised learning module 130 is configured to align the cluster labels of the current iteration with the cluster labels of the previous iteration to obtain the optimized cluster labels, the self-supervised learning module 130 is specifically configured to: obtain a reference cluster center of each reference label according to a feature vector corresponding to the reference label; obtain an alignment cluster center of each alignment label according to a feature vector of training data corresponding to the alignment label; determine a second mapping relationship according to each reference cluster center and each alignment cluster center, the second mapping relationship representing a one-to-one mapping relationship between all reference labels and all alignment labels; and update, according to the second mapping relationship, the alignment label of the training data to a reference label corresponding to the alignment label to obtain the optimized cluster label of the training data.

[0190] Please refer to Figure 10 , Figure 10 The block schematic diagram of the intelligent customer service question clustering device 200 provided in the embodiment of the present application is shown.

[0191] The intelligent customer service question clustering device 200 comprises a question acquisition module 210 and a clustering result generation module 220.

[0192] The question acquisition module 210 is configured to acquire intelligent customer service questions to be clustered.

[0193] The clustering result generation module 220 is configured to cluster the intelligent customer service questions by using the model obtained by the model training method provided in the embodiment, to obtain clustering results of the intelligent customer service questions.

[0194] The intelligent customer service question clustering device 200 provided in the embodiment clusters the intelligent customer service questions by using the clustering feature representation model trained by the model training method provided in the embodiment, and can realize accurate clustering and further discover new user intents in time and accurately according to the accurate clustering results.

[0195] The embodiment of the present application further provides a block schematic diagram of an electronic device 10, and the model training method and the intelligent customer service question clustering method in the foregoing embodiments are applied to the electronic device 10. It should be noted that the model training method and the intelligent customer service question clustering method can be applied to the same electronic device 10 or different electronic devices 10. Please refer to Figure 11 , Figure 11 The block schematic diagram of the electronic device 10 provided in the embodiment of the present application is shown, and the electronic device 10 comprises a processor 11, a memory 12 and a bus 13, and the processor 11, the memory 12 and the bus 13 are connected.

[0196] The processor 11 can be an integrated circuit chip with a signal processing capability. In the implementation process, each step of the model training method or the intelligent customer service question clustering method can be completed by the integrated logic circuit of hardware or the instruction in the form of software in the processor 11. The processor 11 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP) and the like; and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0197] The memory 12 is configured to store programs, for example, the model training device 100 in the foregoing embodiment or the intelligent customer service question clustering device 200 in the foregoing embodiment. Figure 9 Figure 10 ​The model training device 100 or the intelligent customer service question clustering device 200 respectively comprises at least one software function module which can be stored in the memory 12 in the form of software or firmware or solidified in the operating system (OS) of the electronic device 10. The processor 11 executes the program to realize the model training method or the intelligent customer service question clustering method disclosed in the above embodiments after receiving an execution instruction.

[0198] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to realize the model training method or the intelligent customer service question clustering method as described above.

[0199] To sum up, the embodiments of the present application provide a model training method, an intelligent customer service question clustering method and related devices. The method comprises: inputting a training set into a clustering feature representation model to obtain a feature vector of each training data, wherein the training set comprises labeled training data and unlabeled training data; clustering each feature vector to obtain a clustering cluster and a cluster label corresponding to each clustering cluster; when the clustering cluster does not satisfy a clustering condition, iteratively optimizing each cluster label according to a label set and / or each cluster label, wherein the label set comprises a real label of each labeled training data in the training set; training the clustering feature representation model based on the training data and the optimized cluster label to obtain a trained clustering feature representation model. In this embodiment, the labeled annotation data and the unlabeled annotation data are fused to train the clustering feature representation model, so that the labeled annotation data and the unlabeled annotation data can be fully utilized, the clustering effect of the trained clustering feature representation model is better, and finally the intelligent customer service question can be accurately clustered by using the clustering feature representation model, and new user intents can be discovered in time according to the clustering result.

[0200] The above merely provides specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A model training method, characterized in that, The method comprises: inputting a training set into a clustering feature representation model to obtain a feature vector of each training data, the training set comprising labeled training data and unlabeled training data, the training data being in the form of text questions; clustering each feature vector to obtain a clustering cluster and a cluster label corresponding to each clustering cluster; when the clustering cluster does not satisfy a clustering condition, iteratively optimizing each cluster label according to a label set and / or each cluster label, the label set comprising a true label of each labeled training data in the training set; training the clustering feature representation model based on the training data and the optimized cluster label to obtain a trained clustering feature representation model; the step of iteratively optimizing each cluster label according to the label set and / or each cluster label further comprises: taking the cluster label of the last iteration as a reference label and taking the cluster label of the current iteration as a label to be aligned; when the current optimization is not the first iteration, updating the label to be aligned of the training data to the reference label corresponding to the label to be aligned according to a one-to-one mapping relationship between all reference labels and all labels to be aligned to obtain the optimized cluster label of the training data, wherein the cluster label of the labeled training data in the cluster label of the current iteration is specifically an optimized cluster label obtained by modifying the cluster label corresponding to the labeled training data according to the true label of the labeled training data. 2.The model training method of claim 1, wherein, The step of iteratively optimizing each cluster label according to the label set and / or each cluster label comprises: when the current optimization is the first iteration, modifying the cluster label corresponding to the labeled training data according to the true label of the labeled training data to obtain the optimized cluster label. 3.The model training method of claim 1 or 2, wherein, The step of training the clustering feature representation model based on the training data and the optimized cluster label to obtain a trained clustering feature representation model comprises: performing parameter optimization on the clustering feature representation model according to the training data, the cluster label corresponding to the unlabeled training data and the optimized cluster label to obtain an optimized clustering feature representation model; inputting the training set into the optimized clustering feature representation model to obtain an optimized feature vector of each training data; clustering each optimized feature vector to obtain an optimized clustering cluster and an optimized cluster label corresponding to each optimized clustering cluster; when the optimized clustering cluster satisfies the clustering condition, the optimized clustering feature representation model is the trained clustering feature representation model. 4.The model training method of claim 3, wherein, The method further comprises: for each feature vector, calculating an intra-cluster distance and an inter-cluster distance of the feature vector, the intra-cluster distance being an average distance of the feature vector and other feature vectors in a target clustering cluster to which the feature vector belongs, and the inter-cluster distance being a minimum value of an average distance of the feature vector and all feature vectors in each other clustering cluster except the target clustering cluster; calculating a silhouette coefficient according to the intra-cluster distance and the inter-cluster distance of each feature vector; and calculating a silhouette coefficient according to the intra-cluster distance and the inter-cluster distance of each feature vector. If the silhouette coefficient converges, it is determined that the clustering cluster satisfies a clustering condition; If the silhouette coefficient does not converge, it is determined that the clustering cluster does not satisfy the clustering condition. 5.The model training method of claim 2, wherein, The step of modifying the cluster label corresponding to the labeled training data according to the true label of the labeled training data to obtain the optimized cluster label comprises: obtaining a category center of each true label according to a feature vector corresponding to each true label; obtaining a cluster center of each cluster label according to a feature vector corresponding to each cluster label; determining a first mapping relationship according to each category center and each cluster center, the first mapping relationship representing a one-to-one mapping relationship between all true labels and all cluster labels; modifying the cluster label of each labeled training data to the cluster label corresponding to the true label of each labeled training data according to the first mapping relationship. 6.The model training method of claim 1, wherein, The step of updating the to-be-aligned label of the training data to the reference label corresponding to the to-be-aligned label according to a one-to-one mapping relationship between all reference labels and all to-be-aligned labels to obtain the optimized cluster label of the training data comprises: obtaining a reference cluster center of each reference label according to a feature vector corresponding to each reference label; obtaining a to-be-aligned cluster center of each to-be-aligned label according to a feature vector of the training data corresponding to each to-be-aligned label; determining a second mapping relationship according to each reference cluster center and each to-be-aligned cluster center, the second mapping relationship representing a one-to-one mapping relationship between all reference labels and all to-be-aligned labels; updating the to-be-aligned label of the training data to the reference label corresponding to the to-be-aligned label according to the second mapping relationship to obtain the optimized cluster label of the training data.

7. An intelligent customer service question clustering method, characterized by, The method comprises: obtaining intelligent customer service questions to be clustered; performing clustering on the intelligent customer service questions by using a model obtained by the model training method according to any one of claims 1 to 6 to obtain a clustering result of the intelligent customer service questions.

8. A model training apparatus, comprising: The device comprises: a vector generation module configured to input a training set into a clustering feature representation model to obtain a feature vector of each training data, the training set comprising labeled training data and unlabeled training data, and the training data being in the form of text questions; a clustering module configured to cluster each feature vector to obtain a clustering cluster and a cluster label corresponding to each clustering cluster; a self-supervised learning module configured to, when the clustering cluster does not satisfy a clustering condition, iteratively optimize each cluster label according to a label set and / or each cluster label, the label set comprising true labels of each labeled training data in the training set, and train the clustering feature representation model based on the training data and the optimized cluster label to obtain a trained clustering feature representation model; the self-supervised learning module is specifically configured to take the cluster label of a previous iteration as a reference label and take the cluster label of a current iteration as a to-be-aligned label; When the optimization is performed in the current optimization non-first iteration, the to-be-aligned labels of the training data are updated to the reference labels corresponding to the to-be-aligned labels according to one-to-one mapping relationships between all the reference labels and all the to-be-aligned labels, to obtain optimized cluster labels of the training data, wherein the cluster label of the training data with a label in the current iteration is specifically: an optimized cluster label obtained by modifying the cluster label corresponding to the training data with the label according to the real label of the training data with the label.

9. An intelligent customer service question clustering apparatus, characterized by, The device comprises: a question obtaining module configured to obtain intelligent customer service questions to be clustered; a cluster result generating module configured to cluster the intelligent customer service questions by using a model trained by the model training method according to any one of claims 1-6, to obtain a cluster result of the intelligent customer service questions.

10. An electronic device, comprising: comprising a processor and a memory, the memory being configured to store a program, and the processor being configured to implement the model training method according to any one of claims 1-6 and / or the intelligent customer service question clustering method according to claim 7 when executing the program.

11. A computer readable storage medium, characterized in that, a computer program is stored thereon, and the computer program is configured to implement the model training method according to any one of claims 1-6 and / or the intelligent customer service question clustering method according to claim 7 when executed by a processor.

Images