Classification model training method, clustering method and electronic device

Abstract

The present disclosure provides a classification model training method, a clustering method and an electronic device, which relate to the field of artificial intelligence technology, especially big data, deep learning, intent recognition and other technical fields. The specific scheme is: obtain a first unlabeled text data set; input the first unlabeled text data set into the first classification model for iterative training to obtain a target classification model; not only the first classification model is iteratively trained by the first unlabeled text data set, but also the loss value of the first classification model is calculated by the predicted intent category label and the reference intent category label of the first unlabeled text data set during the iterative training process, and the predicted intent category label and the reference intent category label are obtained based on the first classification model during the iterative training process, that is, the predicted intent category label and the reference intent category label are obtained based on the first classification model during the iterative training process to realize the training of the model, so that the effect of model training can be improved.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technologies such as big data, deep learning, and intent recognition, and in particular to a classification model training method, a clustering method, and an electronic device. Background Art

[0002] With the development of artificial intelligence, intelligent technologies such as spatiotemporal big data perception and artificial intelligence customer service are becoming more and more integrated into people's daily lives. Whether it is intent recognition in spatiotemporal big data or multi-round dialogue scenarios of artificial intelligence customer service, it is necessary to recognize the input intent and then make relevant operations. In the process of text intent processing, intent clustering is an important part.

[0003] In the clustering process, semi-supervised clustering is often used. In the semi-supervised clustering method, the model needs to be trained first, that is, model training is an important part of it. Currently, in the process of training the model, pre-labeled annotated text data is used as input into the model for training. Summary of the invention

[0004] The present disclosure provides a classification model training method, a clustering method and an electronic device.

[0005] In a first aspect, an embodiment of the present disclosure provides a classification model training method, the method comprising:

[0006] Obtain a first unlabeled text dataset;

[0007] Inputting the first unlabeled text data set into the first classification model for iterative training to obtain a target classification model;

[0008] Among them, during the iterative training process, the loss value of the first classification model is calculated through the predicted intent category label and the reference intent category label of the first unlabeled text data set, and the predicted intent category label and the reference intent category label are obtained based on the first classification model during the iterative training process.

[0009] In the classification model training method of the present embodiment, not only is the first classification model iteratively trained using the first unlabeled text data set, but also during the iterative training process, the loss value used by the first classification model is calculated using the predicted intent category label of the first unlabeled text data set and the reference intent category label of the first unlabeled text data set. The predicted intent category label and the reference intent category label are obtained based on the first classification model during the iterative training process. That is, during the iterative training process, the predicted intent category label and the reference intent category label can be obtained based on the first classification model, and the loss value is calculated using the predicted intent category label and the reference intent category label to achieve model training. In this way, the effect of model training can be improved.

[0010] In a second aspect, an embodiment of the present disclosure provides a clustering method, wherein the method includes:

[0011] Using the target language model in the target classification model to perform feature extraction on the text data set to be clustered, to obtain a feature vector set of the text data set to be clustered;

[0012] Performing intent clustering on the text data set to be clustered based on the feature vector set to obtain N clustering clusters, where N is a positive integer;

[0013] Among them, the loss value used in the target classification model training is calculated through the predicted intent category label and the reference intent category label of the first unlabeled text data set.

[0014] In a third aspect, an embodiment of the present disclosure provides a classification model training device, the device comprising:

[0015] A first acquisition module, used to acquire a first unlabeled text dataset;

[0016] A first training module is used to input the first unlabeled text data set into the first classification model for iterative training to obtain a target classification model;

[0017] Among them, during the iterative training process, the loss value of the first classification model is calculated through the predicted intent category label and the reference intent category label of the first unlabeled text data set, and the predicted intent category label and the reference intent category label are obtained based on the first classification model during the iterative training process.

[0018] In a fourth aspect, an embodiment of the present disclosure provides a clustering device, wherein the device includes:

[0019] A feature extraction module, used to extract features of the text data set to be clustered using the target language model in the target classification model, to obtain a feature vector set of the text data set to be clustered;

[0020] A clustering module, used for performing intention clustering on the text data set to be clustered based on the feature vector set to obtain N cluster clusters, where N is a positive integer;

[0021] Among them, the loss value used in the target classification model training is calculated through the predicted intent category label and the reference intent category label of the first unlabeled text data set.

[0022] In a fifth aspect, an embodiment of the present disclosure further provides an electronic device, including:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the classification model training method provided in the first aspect of the present disclosure.

[0026] In a sixth aspect, an embodiment of the present disclosure further provides an electronic device, including:

[0027] at least one processor; and

[0028] a memory communicatively connected to the at least one processor; wherein,

[0029] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the clustering method provided by the second aspect of the present disclosure.

[0030] In the seventh aspect, an embodiment of the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the classification model training method provided in the first aspect or the clustering method provided in the second aspect of the present disclosure.

[0031] In an eighth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the classification model training method provided in the first aspect or the clustering method provided in the second aspect of the present disclosure.

[0032] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0034] Figure 1 It is a flowchart of a classification model training method according to an embodiment of the present disclosure;

[0035] Figure 2 is a schematic diagram of a clustering method according to an embodiment of the present disclosure;

[0036] Figure 3 It is a principle diagram of a classification model training method and a clustering method according to an embodiment of the present disclosure;

[0037] Figure 4It is a principle diagram of training an initial classification model to obtain a first classification model in a classification model training method provided by an embodiment of the present disclosure;

[0038] Figure 5 It is a schematic diagram of training a first classification model to obtain a target classification model in a classification model training method provided by an embodiment of the present disclosure;

[0039] Figure 6 It is a flowchart of generating pseudo labels by clustering in a classification model training method provided by an embodiment of the present disclosure;

[0040] Figure 7 is a structural diagram of a classification model training device according to an embodiment of the present disclosure;

[0041] Figure 8 is a structural diagram of a clustering device according to an embodiment of the present disclosure;

[0042] Fig. 9 It is a block diagram of an electronic device used to implement the classification model training method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0044] like Figure 1 As shown, according to an embodiment of the present disclosure, the present disclosure provides a classification model training method, the method comprising:

[0045] Step S101: Acquire a first unlabeled text dataset.

[0046] An unlabeled text dataset can be understood as a dataset without intent category labels, which may include multiple unlabeled text data for model training. It can be understood that a labeled text dataset is a dataset with intent category labels, that is, it includes not only multiple labeled text data, but also the intent category labels corresponding to the multiple labeled text data.

[0047] Step S102: inputting the first unlabeled text data set into the first classification model for iterative training to obtain a target classification model;

[0048] Among them, during the iterative training process, the loss value of the first classification model is calculated through the predicted intent category labels and reference intent category labels of the first unlabeled text data set, and the predicted intent category labels and reference intent category labels are obtained based on the first classification model during the iterative training process.

[0049] Unlabeled text data contains more semantic knowledge. In this embodiment, the first unlabeled text data set is used to train the first classification model to obtain the target classification model, so that the target classification model can better learn the semantic knowledge of the first unlabeled text data set. The first classification model corresponds to a loss function. There are many types of loss functions, which are not limited in this embodiment. Model training can be understood as minimizing the loss function to make the model reach a convergence state. When the model does not converge, the parameters of the model can be continuously adjusted to obtain the output based on the model again. The model training is completed by continuous iterative training until the model converges and the loss function reaches a minimum value. In the iterative training process, each iterative training can calculate the value of the loss function used by the first classification model according to the predicted intent category label of the first unlabeled text data set by the first classification model and the reference intent category label obtained based on the first classification model, that is, a loss value is calculated, that is, in the iterative training process, the value of the loss function of the first classification model of this embodiment is calculated by the predicted intent category label of the first unlabeled text data set and the reference intent category label, and the predicted intent category label and the reference intent category label are obtained based on the first classification model during the iterative training process, and there is no need to manually annotate data in advance to obtain annotated data for training. It should be noted that the first classification model includes a first language model and a first classification layer connected in sequence, and the target classification model may include a target language model and a target classification layer connected in sequence, the target language model is a model trained by the first language model, and the target classification layer is a classification layer trained by the first classification layer.

[0050] In the classification model training method of the present embodiment, not only is the first classification model iteratively trained using the first unlabeled text data set, but also during the iterative training process, the loss value used by the first classification model is calculated using the predicted intent category label of the first unlabeled text data set and the reference intent category label of the first unlabeled text data set. The predicted intent category label and the reference intent category label are obtained based on the first classification model during the iterative training process. That is, during the iterative training process, the predicted intent category label and the reference intent category label can be obtained based on the first classification model, and the loss value is calculated using the predicted intent category label and the reference intent category label to achieve model training. In this way, the effect of model training can be improved.

[0051] In one embodiment, a predicted intent category label is obtained by performing intent classification on a first unlabeled text dataset using a first classification model, a reference intent category label is obtained by performing intent category labels of multiple clusters of the first unlabeled text dataset, the multiple clusters are obtained by clustering the first unlabeled text dataset based on a feature vector set, and the feature vector set is obtained by performing feature extraction on the first unlabeled text dataset using the first classification model.

[0052] That is, during the iterative training process, the first unlabeled text data set is input into the first classification model, and the predicted intent category label can be output through the first classification model. Moreover, the first classification model can be used to extract features from the input first unlabeled text data set to obtain a feature vector set of the first unlabeled data set. The first unlabeled text data set is clustered based on the feature vector set obtained by the first classification model using a clustering algorithm (not limited in this embodiment) to obtain multiple clusters of the first unlabeled text data set and intent category labels of multiple clusters. For each cluster, the intent category label of the cluster can be used as a reference intent category label for the text data in the cluster, thereby labeling each text data in the first unlabeled text data set with a corresponding reference intent category label to obtain a reference intent category label for the first unlabeled text data set. During each iterative training process, the loss value of the first classification model is calculated using the predicted intent category label and the reference intent category label of the first unlabeled text data set obtained in the overshoot of this iterative training. During the iterative training process, as the first classification model is continuously updated (optimized), the feature vector generated by the first classification model for the first unlabeled text data set is continuously updated, and then the clustering results obtained through clustering are continuously updated, then the reference intent category is continuously updated, and the predicted intent category label generated by the first classification model is also continuously updated. In this way, through multiple iterative training, clustering and classification promote each other until the model converges and model training is realized, thereby improving the effect of model training, so that the obtained target classification model can better extract the semantic knowledge of text data.

[0053] In one embodiment, before inputting the first unlabeled text dataset into the first classification model for iterative training to obtain the target classification model, the method further includes:

[0054] Acquire a second unlabeled text dataset and a first labeled text dataset;

[0055] Inputting the second unlabeled text data set into the pre-trained language model for training to obtain an intermediate language model;

[0056] An initial classification model is constructed based on the intermediate language model and the initial classification layer, and the first annotated text data set is input into the initial classification model for training to obtain a first classification model, wherein the first classification model includes a first language model and a first classification layer connected in sequence.

[0057] In this embodiment, the second unlabeled text data set can be used to input the pre-trained language model for training to obtain an intermediate language model, that is, based on the pre-trained language model, the second unlabeled text data set is used for training to enhance the performance of the intermediate language model. Then, the intermediate language model and the initial classification layer are used to construct the initial classification model. It can be understood that the input of the initial classification layer is connected to the output of the intermediate language model, and the initial classification model is trained by inputting the first labeled text data set to obtain the first classification model. The first language model is the model after the intermediate language model is trained, and the first classification layer is the classification layer after the initial classification layer is trained. At this stage, the prior knowledge in the first labeled text data set can be injected into the classification model to enhance the performance of the first classification model. Then, the first unlabeled text data set is input into the first classification model for iterative training to obtain the target classification model. In this way, the model training effect can be further improved, and the performance of the target classification model obtained can be improved.

[0058] It should be noted that the predicted intent category label is output through the first classification layer in the first classification model, and the feature vector set of the first unlabeled text data set is obtained by performing feature extraction on the first unlabeled text data set through the first language model in the first classification model.

[0059] As an example, the first unlabeled text dataset may be the first unlabeled text dataset of the target domain (specific domain), the second unlabeled text dataset may be the second unlabeled text dataset of the target domain, and the first labeled text dataset may also be the first labeled text dataset of the target domain.

[0060] In one embodiment, during the M+1th iterative training, the intent category labels of the multiple clusters of the first unlabeled text dataset are category labels aligned with the Mth intent category labels;

[0061] The M-th intent category label is the intent category label of multiple clusters of the first unlabeled text dataset during the M-th iterative training, and M is a positive integer.

[0062] Since the cluster category numbers (intent category labels) clustered between the Mth round and the M+1th round may change, the Hungarian algorithm can be used to align the clusters generated by adjacent rounds of iterations, so that each cluster can have a unique intent category label. It should be noted that the intent category labels of the multiple clusters of the first unlabeled text dataset during the Mth iterative training are the category labels aligned with the intent category labels of the multiple clusters of the first unlabeled text dataset during the M-1th iterative training, and M-1 is a positive integer.

[0063] During the iterative training process of this embodiment, the intent category labels of the multiple clusters of the first unlabeled text data set used in the M+1th iterative training process are category labels that are aligned with the Mth intent category labels. In this way, the accuracy of the intent category labels can be improved, and they can be subsequently used to calculate the loss value to implement model training, thereby improving the effect of model training.

[0064] like Figure 2 As shown, according to an embodiment of the present disclosure, the present disclosure also provides a clustering method, wherein the method includes:

[0065] Step 201: Using the target language model in the target classification model, feature extraction is performed on the text data set to be clustered to obtain a feature vector set of the text data set to be clustered.

[0066] The text dataset to be clustered is a text dataset that needs to be clustered, which can be understood as an unlabeled text dataset to be clustered.

[0067] Step 202: performing intent clustering on the text data set to be clustered based on the feature vector set to obtain N clusters, where N is a positive integer;

[0068] Among them, the loss value used in the target classification model training is calculated through the predicted intent category label and the reference intent category label of the first unlabeled text dataset.

[0069] It should be noted that the target classification model is obtained by training the first classification model using the first unlabeled text dataset. In the process of training the target classification model, the loss value used is calculated through the predicted intent category label and the reference intent category label of the first unlabeled text dataset.

[0070] In the classification model training method of this embodiment, the loss value used for training the target classification model is calculated based on the predicted intent category labels and reference intent category labels of the first unlabeled text dataset. By using the target language model in the target classification model to extract features from the text dataset to be clustered, the accuracy of the feature vector set of the text dataset to be clustered can be improved. Based on the feature vector set, intent clustering is performed on the text dataset to be clustered, and N clustering clusters are obtained, thereby improving the clustering effect and the clustering accuracy.

[0071] In one embodiment, after performing intent clustering on the text dataset to be clustered based on the feature vector set to obtain N clustering clusters, it further includes:

[0072] Segment the text data in the target clustering cluster and filter out the preset word segmentation, to obtain the target word segmentation set of the target clustering cluster, where the target clustering cluster is any one of the N clustering clusters;

[0073] Calculate the term frequency-inverse document frequency value of the word segmentation in the target word segmentation set;

[0074] Determine the candidate keywords of the target word segmentation set according to the term frequency-inverse document frequency value of the word segmentation in the target word segmentation set;

[0075] Filter the text data in the target clustering cluster according to the candidate keywords.

[0076] The preset word segmentation can also be understood as common stop words, that is, stop words can be removed. The preset word segmentation can be pre-configured. For example, the preset word segmentation can include "of", "some", "first", etc. For each of the N clustering clusters, the above process is performed for filtering, determining candidate keywords, and filtering the text data in the clustering cluster. It should be noted that the term frequency-inverse document frequency value is the TF-IDF (term frequency–inverse document frequency) value, which is the product of the term frequency (TF, term frequency) and the inverse document frequency value (IDF, inverse document frequency).

[0077] In this embodiment, the term frequency-inverse document frequency value of the word segmentation in the target word segmentation set of the target clustering cluster can be calculated. Through the term frequency-inverse document frequency value, the candidate keywords of the target word segmentation set are determined, and the text data in the target clustering cluster is filtered according to the candidate keywords, that is, denoising is performed on the clustering cluster to improve the accuracy of the clustering cluster.

[0078] In one embodiment, determining the candidate keywords of the target word segmentation set according to the term frequency-inverse document frequency value of the word segmentation in the target word segmentation set includes:

[0079] Determine n words in the target word set as candidate keywords, where n is a positive integer, and the word frequency inverse file frequency values ​​of the n words are greater than the word frequency inverse file frequency values ​​of the remaining words, and the remaining words are the words in the target word set except the n words.

[0080] That is, in this embodiment, the first n word segments with the frequency inverse file frequency values ​​are taken as candidate keywords for the cluster to improve the accuracy of the candidate keywords. Subsequently, the text data in the target cluster is filtered according to the candidate keywords to improve the filtering accuracy, thereby improving the accuracy of the cluster.

[0081] In one embodiment, filtering text data in a target cluster according to candidate keywords includes:

[0082] The target text data in the target cluster is filtered, wherein the number of target keywords in the target text data is less than a preset number, and the target keyword is a candidate keyword.

[0083] That is, if a certain text data of the target cluster includes at least a preset number of keywords among the candidate keywords, then the text data does not need to be filtered and remains in the target cluster; if a certain text data of the target cluster includes less than a preset number of keywords among the candidate keywords, then the text data is filtered and deleted from the target cluster, thereby filtering the noise data of the target cluster and improving the accuracy of the cluster.

[0084] As an example, before filtering the text data in the target cluster according to the candidate keywords, the method may further include: determining synonyms corresponding to the candidate keywords, adding the synonyms to the candidate keywords, and updating the candidate keywords. Subsequently, filtering the text data in the target cluster according to the updated candidate keywords improves the filtering effect.

[0085] In one embodiment, the target classification model is trained in the following manner:

[0086] Obtain a first unlabeled text dataset;

[0087] Inputting the first unlabeled text data set into the first classification model for iterative training to obtain a target classification model;

[0088] Among them, during the iterative training process, the loss value of the first classification model is calculated through the predicted intent category labels and reference intent category labels of the first unlabeled text data set, and the predicted intent category labels and reference intent category labels are obtained based on the first classification model during the iterative training process.

[0089] In one embodiment, a predicted intent category label is obtained by performing intent classification on a first unlabeled text dataset through a first classification model, a reference intent category label is obtained through the intent category labels of multiple clusters of the first unlabeled text dataset, the multiple clusters are obtained by clustering the first unlabeled text dataset based on a feature vector set of the first unlabeled text dataset, and the feature vector set of the first unlabeled text dataset is obtained by performing feature extraction on the first unlabeled text dataset through the first classification model.

[0090] It should be noted that the target classification model used in the clustering method of the present disclosure can be trained according to the classification model training method in the above embodiment, which will not be repeated here.

[0091] The above classification model training method and clustering method are specifically described below with a specific embodiment.

[0092] The method proposed in the present disclosure is mainly formed by combining a large-scale pre-trained language model and a clustering algorithm, and is mainly divided into four stages: the first stage is to use a large amount of unlabeled domain data in a specific business field to perform domain pre-training on the pre-trained language module. This stage allows the model to learn a large amount of domain knowledge, thereby having a certain gain in downstream classification and clustering tasks; the second stage is to build an initial classification model based on the intermediate language model obtained by domain training the pre-trained language model in the first stage, and use limited labeled data to train the initial classification model to obtain a first classification model. The purpose of this step is to inject prior knowledge in the labeled data into the model, thereby guiding the subsequent clustering of intent topics; the third stage is to use a large amount of unlabeled data for clustering, and at the same time, combine the idea of ​​self-supervision to use the pseudo-labels after clustering (i.e., the intent category labels of the clustering times) to obtain the value of the loss function of the first classification model, and use the value of this loss function to continuously adjust the first classification model, so that the intermediate language model in the first classification model produces a more reasonable feature vector representation, thereby adjusting the clustering clusters; the first three stages are training stages, such as Figure 3 As shown by the solid line in , the training is completed and the target classification model is obtained. The fourth stage is the prediction stage, as shown in Figure 3 As shown by the dotted line in , a large number of unlabeled data sets to be clustered are input into the target classification model, and features are extracted through the target language model in the trained target classification model. After clustering using the extracted feature vector set, the keywords based on the TF-IDF value are filtered in each cluster to obtain the final cluster output. Each cluster represents a different intent type and contains all texts with the same intent category in the text data set to be clustered.

[0093] For stage 1: in-domain pre-training:

[0094] With the development of large-scale pre-trained language models, most natural language processing tasks will be fine-tuned on the pre-trained language models. Although this simple fine-tuning can achieve good results, it cannot make further breakthroughs. Therefore, the method of re-pretraining the pre-trained model with in-domain data has become a simple and effective strategy. Generally speaking, pre-trained language models such as ERNIE and BERT are pre-trained on large-scale open domain datasets. They learn general knowledge in open domains. Although these prior knowledge has a certain effect on intent clustering, directly using these pre-trained models for fine-tuning in specified domains under specific business scenarios often does not work well. Therefore, in the method proposed in the present invention, first, a dataset in a specific domain is used to pre-train the domain knowledge again on the basis of the pre-trained language model, so that the model can learn the prior information in the domain, and then the model is fine-tuned. Figure 3 As shown in the figure, the second unlabeled data set will be trained by the pre-trained language model, and the pre-trained language model will be continuously optimized and adjusted by the loss function of the pre-trained language model. For example, in the complaint situation awareness scenario, the general knowledge learned by the model may generally be limited to distinguishing "complaint" and "non-complaint" sentences. However, after in-domain pre-training, the model can learn the difference between different complaint types in the complaint scenario, such as "hotel environmental sanitation complaints" and "hotel service complaints", making subsequent clustering more accurate.

[0095] For stage 2: intent classification pre-training:

[0096] Generally speaking, semi-supervised learning methods are more effective than completely unsupervised methods. Therefore, the method proposed in this disclosure also uses partially labeled data combined with a large amount of unlabeled data to perform semi-supervised clustering of intent. Figure 4 As shown, firstly, in a specific field, active learning and other methods are combined to annotate the data in the domain. After obtaining the first annotated data set, an initial classification model of intent is formed by combining the intermediate language model and the initial classification layer in the specific field. This initial classification model is used to train on the first annotated data, and the initial classification model is fine-tuned to obtain the first classification model, so that the first classification model learns the intent category knowledge in the first annotated data, so that the feature vector generated by the model has a certain degree of distinction at the user annotation granularity, which is more conducive to clustering. For example, the sentence structure between "When can the English Level 6 test results be checked?" and "How to check the English Level 6 test results?" is roughly the same, but their intents are different. One is the query time and the other is the query method. Through the annotation information of the annotated data, the model can learn the difference in intent, thereby improving the accuracy of clustering.

[0097] For stage 3: self-supervised clustering:

[0098] After the first classification model is obtained through training, the first classification model is trained again using the first unlabeled data set, and the data after de-labeling of the above-mentioned first labeled text data set and a large amount of unlabeled data (which may include at least part of the second unlabeled text data set) are clustered to form a first unlabeled text data set. The purpose of doing this is that since these first annotations have been used for pre-training of intent classification, adding them to the data set of this stage can fully utilize the value of these labeled data and further guide the generation of clustering clusters. In the clustering process, the method proposed in the present disclosure combines self-supervised learning with alternating training, and can generate pseudo-labels from the unlabeled data set and learn by itself, thereby mining the supervision information in the data set. Figure 5-6 As shown, the method first clusters the first unlabeled text data set to form multiple clusters, and then forms multiple clusters according to the threshold. Different intent categories are formed between different clusters, and these categories are used as pseudo labels of the text data in the cluster for self-supervised model training. Specifically, the second unlabeled text data set will give each text data a predicted intent category label after passing through the first classification model. The pseudo label and predicted category label obtained by the above method are used to obtain the value of the loss function of the first classification model. The first classification model parameters are optimized using the value of this loss function. The updated first classification model will subsequently generate a new feature vector representation based on the first unlabeled text data set, and then the clustering process will generate new pseudo labels. This cycle is repeated and mutually promoted until the model converges to obtain the target classification model. Since the cluster category numbers clustered between the Mth round and the M+1th round may change, the Hungarian algorithm is used here to align the different clusters generated by the front and rear round iterations, so that each cluster can have a unique pseudo label.

[0099] For stage 4: model clustering prediction:

[0100] After the model training is completed and the target classification is obtained, the target language model is used to extract features of the text dataset to be clustered, and the clustering algorithm is used to perform clustering prediction on the text dataset to be clustered according to the extracted feature vector set to form N clusters, each of which contains at least one text data, such as Figure 3As shown by the dotted line in . In order to make the text data in each cluster as close as possible to the intent category of the cluster, the TF-IDF-based keyword filtering technology can be used to denoise the cluster. The specific method is: for each cluster, first perform word segmentation and remove stop words, then use TF-IDF to calculate the TF-IDF value of each word in the cluster, take the top-n words as candidate keywords for the cluster, use these candidate keywords to find the corresponding synonyms using the synonym dictionary, add them to the above candidate keywords to form new candidate keywords, and finally use the candidate keywords of the cluster to filter the noise data of the text data in the cluster, and form the final cluster after filtering.

[0101] The method proposed in the present disclosure combines pre-training and keyword filtering in the field. The method is based on a semi-supervised clustering algorithm and combines a domain pre-training method at the same time. It can make full use of the domain knowledge of a large amount of unlabeled data in the field to enhance the feature representation of input text data. The method also uses the method of alternating training of classification tasks and clustering tasks to inject the prior knowledge of labeled data into the feature vector representation to improve the clustering effect. In addition, the method also uses a self-supervised strategy to combine the pseudo-labels clustered and the predicted intent categories generated by the classification model, so that the classification task and the clustering task promote each other and iterate continuously, thereby generating better clustering clusters. Finally, after the clustering is completed, a TFIDF-based keyword filtering process is added to filter and screen the more obvious noise data in each clustering cluster, thereby improving the accuracy of intent clustering, improving the efficiency of intent induction, and reducing labor costs. For example, it is currently used in user intent recognition of spatiotemporal big data, and the original accuracy can be improved by at least 8%.

[0102] like Figure 7 As shown, according to an embodiment of the present disclosure, the present disclosure further provides a classification model training device 700, the device comprising:

[0103] A first acquisition module 701 is used to acquire a first unlabeled text dataset;

[0104] A first training module 702 is used to input the first unlabeled text data set into the first classification model for iterative training to obtain a target classification model;

[0105] Among them, during the iterative training process, the loss value of the first classification model is calculated through the predicted intent category labels and reference intent category labels of the first unlabeled text data set, and the predicted intent category labels and reference intent category labels are obtained based on the first classification model during the iterative training process.

[0106] In one embodiment, a predicted intent category label is obtained by performing intent classification on a first unlabeled text dataset using a first classification model, a reference intent category label is obtained by performing intent category labels of multiple clusters of the first unlabeled text dataset, the multiple clusters are obtained by clustering the first unlabeled text dataset based on a feature vector set, and the feature vector set is obtained by performing feature extraction on the first unlabeled text dataset using the first classification model.

[0107] In one embodiment, the apparatus 700 further includes:

[0108] A second acquisition module, used to acquire a second unlabeled text dataset and a first labeled text dataset;

[0109] A second training module is used to input the second unlabeled text data set into the pre-trained language model for training to obtain an intermediate language model;

[0110] The third training module is used to build an initial classification model based on the intermediate language model and the initial classification layer, and input the first annotated text data set into the initial classification model for training to obtain a first classification model, wherein the first classification model includes a first language model and a first classification layer connected in sequence.

[0111] In one embodiment, during the M+1th iterative training, the intent category labels of the multiple clusters of the first unlabeled text dataset are category labels aligned with the Mth intent category labels;

[0112] The M-th intent category label is the intent category label of multiple clusters of the first unlabeled text dataset during the M-th iterative training, and M is a positive integer.

[0113] The classification model training device of each of the above embodiments is a device for implementing the classification model training method of each of the above embodiments, and the technical features and technical effects correspond to each other, which will not be repeated here.

[0114] like Figure 8 As shown, according to an embodiment of the present disclosure, the present disclosure further provides a clustering device 800, the device 800 comprising:

[0115] A feature extraction module 801 is used to extract features of the text data set to be clustered using a target language model in a target classification model to obtain a feature vector set of the text data set to be clustered;

[0116] A clustering module 802 is used to perform intent clustering on the text data set to be clustered based on the feature vector set to obtain N clusters, where N is a positive integer;

[0117] Among them, the loss value used in the target classification model training is calculated through the predicted intent category label and the reference intent category label of the first unlabeled text dataset.

[0118] In one embodiment, the apparatus 800 further includes:

[0119] A word segmentation set determination module is used to segment the text data in the target cluster and filter the preset word segments to obtain a target word segmentation set of the target cluster, wherein the target cluster is any cluster among the N clusters;

[0120] A calculation module, used to calculate the word frequency inverse file frequency value of the segmented words in the target segmented word set;

[0121] A keyword determination module, used to determine candidate keywords of the target word set according to the word frequency inverse document frequency value of the word in the target word set;

[0122] The filtering module is used to filter the text data in the target cluster according to the candidate keywords.

[0123] In one embodiment, determining candidate keywords of the target word set according to the word frequency inverse document frequency value of the word in the target word set includes:

[0124] Determine n words in the target word set as candidate keywords, where n is a positive integer, and the word frequency inverse file frequency values ​​of the n words are greater than the word frequency inverse file frequency values ​​of the remaining words, and the remaining words are the words in the target word set except the n words.

[0125] In one embodiment, filtering text data in a target cluster according to candidate keywords includes:

[0126] The target text data in the target cluster is filtered, wherein the number of target keywords in the target text data is less than a preset number, and the target keyword is a candidate keyword.

[0127] In one embodiment, the target classification model is trained in the following manner:

[0128] Obtain a first unlabeled text dataset;

[0129] Inputting the first unlabeled text data set into the first classification model for iterative training to obtain a target classification model;

[0130] Among them, during the iterative training process, the loss value of the first classification model is calculated through the predicted intent category labels and reference intent category labels of the first unlabeled text data set, and the predicted intent category labels and reference intent category labels are obtained based on the first classification model during the iterative training process.

[0131] In one embodiment, a predicted intent category label is obtained by performing intent classification on a first unlabeled text dataset through a first classification model, a reference intent category label is obtained through the intent category labels of multiple clusters of the first unlabeled text dataset, the multiple clusters are obtained by clustering the first unlabeled text dataset based on a feature vector set of the first unlabeled text dataset, and the feature vector set of the first unlabeled text dataset is obtained by performing feature extraction on the first unlabeled text dataset through the first classification model.

[0132] The clustering devices of the above-mentioned embodiments are devices for implementing the clustering methods of the above-mentioned embodiments, and their technical features and technical effects correspond to each other, which will not be elaborated here.

[0133] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0134] The non-transitory computer-readable storage medium of an embodiment of the present disclosure stores computer instructions, which are used to enable a computer to execute the classification model training method or clustering method provided by the present disclosure.

[0135] The computer program product of the embodiments of the present disclosure includes a computer program, and the computer program is used to enable a computer to execute the classification model training method or clustering method provided by the embodiments of the present disclosure.

[0136] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0137] like Fig. 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 909 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0138] Multiple components in the electronic device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the electronic device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0139] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a classification model training method or a clustering method. For example, in some embodiments, the classification model training method or the clustering method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the classification model training method or the clustering method described above may be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform a classification model training method or a clustering method by any other appropriate means (e.g., by means of firmware). Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip systems (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs, which may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general programmable processor, which may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0141] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0143] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0144] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0145] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0146] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A classification model training method, the method comprising: Obtain a first unlabeled text dataset; Inputting the first unlabeled text data set into the first classification model for iterative training to obtain a target classification model; Wherein, in the Mth iteration of the iterative training, the loss value of the first classification model is calculated by using the predicted intent category label and the reference intent category label of the first unlabeled text dataset obtained in the Mth iteration; The step of obtaining the predicted intent category label of the first unlabeled text dataset in the Mth iteration includes: Inputting the first unlabeled text data set into the first classification model of the Mth iteration for classification, and obtaining a predicted intent category label of the first unlabeled text data set in the Mth iteration; The step of obtaining the reference intent category label of the first unlabeled text dataset in the Mth iteration includes: Perform feature extraction on the input first unlabeled text data set based on the first classification model of the Mth iteration to obtain a feature vector set of the first unlabeled text data set in the Mth iteration; Perform intent clustering on the first unlabeled text dataset based on the feature vector set in the Mth iteration, and obtain multiple clusters of the first unlabeled text dataset in the Mth iteration and intent category labels of the multiple clusters, wherein the intent category label of each cluster is a reference intent category label of the text data in the cluster, and the reference intent category label of the first unlabeled text dataset in the Mth iteration includes the reference intent category label corresponding to each text data annotation in the first unlabeled text dataset in the Mth iteration, where M is a positive integer.

2. The method according to claim 1, wherein: Before inputting the first unlabeled text data set into the first classification model for iterative training to obtain the target classification model, the method further includes: Acquire a second unlabeled text dataset and a first labeled text dataset; Inputting the second unlabeled text data set into a pre-trained language model for training to obtain an intermediate language model; An initial classification model is constructed based on the intermediate language model and the initial classification layer, and the first annotated text data set is input into the initial classification model for training to obtain the first classification model, wherein the first classification model includes a first language model and a first classification layer connected in sequence.

3. The method according to claim 1, wherein: During the M+1th iterative training, the intent category labels of the multiple clusters of the first unlabeled text dataset are category labels aligned with the Mth intent category labels; The M-th intent category label is the intent category label of multiple clusters of the first unlabeled text dataset during the M-th iterative training.

4. A clustering method, wherein: The method comprises: Using the target language model in the target classification model to perform feature extraction on the text data set to be clustered, to obtain a feature vector set of the text data set to be clustered; Performing intent clustering on the text data set to be clustered based on the feature vector set to obtain N clustering clusters, where N is a positive integer; Wherein, the target classification model is trained using the classification model training method described in any one of claims 1-3.

5. The method according to claim 4, wherein: After performing intention clustering on the text data set to be clustered based on the feature vector set to obtain N clusters, the method further includes: Segmenting the text data in the target cluster and filtering the preset segmented words to obtain a target segmented word set of the target cluster, wherein the target cluster is any cluster among the N clusters; Calculate the word frequency inverse file frequency value of the segmented words in the target segmented word set; Determining candidate keywords of the target word set according to the word frequency inverse document frequency value of the word set in the target word set; The text data in the target cluster is filtered according to the candidate keywords.

6. The method according to claim 5, wherein: Determining the candidate keywords of the target word set according to the word frequency inverse file frequency value of the word in the target word set includes: Determine n segmentations in the target segmentation set as the candidate keywords, where n is a positive integer, and the word frequency inverse file frequency values ​​of the n segmentations are greater than the word frequency inverse file frequency values ​​of the remaining segmentations, and the remaining segmentations are the remaining segmentations in the target segmentation set except the n segmentations.

7. The method according to claim 5, wherein: The filtering of the text data in the target cluster according to the candidate keywords includes: The target text data in the target cluster is filtered, wherein the number of target keywords in the target text data is less than a preset number, and the target keyword belongs to the candidate keyword.

8. The method according to claim 4, wherein: The target classification model is trained in the following way: Obtain a first unlabeled text dataset; Inputting the first unlabeled text data set into the first classification model for iterative training to obtain a target classification model; Among them, during the iterative training process, the loss value of the first classification model is calculated through the predicted intent category label and the reference intent category label of the first unlabeled text data set, and the predicted intent category label and the reference intent category label are obtained based on the first classification model during the iterative training process.

9. The method according to claim 8, wherein: The predicted intent category label is obtained by performing intent classification on the first unlabeled text dataset by the first classification model, and the reference intent category label is obtained by the intent category labels of multiple clusters of the first unlabeled text dataset, and the multiple clusters are obtained by clustering the first unlabeled text dataset based on the feature vector set of the first unlabeled text dataset, and the feature vector set of the first unlabeled text dataset is obtained by performing feature extraction on the first unlabeled text dataset by the first classification model.

10. A classification model training device, the device comprising: A first acquisition module, used to acquire a first unlabeled text dataset; A first training module is used to input the first unlabeled text data set into the first classification model for iterative training to obtain a target classification model; in, In the Mth iteration of the iterative training, the loss value of the first classification model is calculated by using the predicted intent category label and the reference intent category label of the first unlabeled text dataset obtained in the Mth iteration; The step of obtaining the predicted intent category label of the first unlabeled text dataset in the Mth iteration includes: Inputting the first unlabeled text data set into the first classification model of the Mth iteration for classification, and obtaining a predicted intent category label of the first unlabeled text data set in the Mth iteration; The step of obtaining the reference intent category label of the first unlabeled text dataset in the Mth iteration includes: Perform feature extraction on the input first unlabeled text data set based on the first classification model of the Mth iteration to obtain a feature vector set of the first unlabeled text data set in the Mth iteration; Perform intent clustering on the first unlabeled text dataset based on the feature vector set in the Mth iteration, and obtain multiple clusters of the first unlabeled text dataset in the Mth iteration and intent category labels of the multiple clusters, wherein the intent category label of each cluster is a reference intent category label of the text data in the cluster, and the reference intent category label of the first unlabeled text dataset in the Mth iteration includes the reference intent category label corresponding to each text data annotation in the first unlabeled text dataset in the Mth iteration, where M is a positive integer.

11. A clustering device, wherein: The device comprises: A feature extraction module, used to extract features of the text data set to be clustered using the target language model in the target classification model, to obtain a feature vector set of the text data set to be clustered; A clustering module, used for performing intention clustering on the text data set to be clustered based on the feature vector set to obtain N cluster clusters, where N is a positive integer; Wherein, the target classification model is trained using the classification model training method described in any one of claims 1-3.

12. The device according to claim 11, wherein Also includes: A word segmentation set determination module, used to segment text data in a target cluster and filter preset word segments to obtain a target word segmentation set of the target cluster, wherein the target cluster is any cluster among the N clusters; A calculation module, used to calculate the word frequency inverse file frequency value of the segmented words in the target segmented word set; A keyword determination module, used to determine candidate keywords of the target word set according to the word frequency inverse file frequency value of the word in the target word set; The filtering module is used to filter the text data in the target cluster according to the candidate keywords.

13. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the classification model training method described in any one of claims 1-3.

14. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the clustering method described in any one of claims 4-9.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the classification model training method described in any one of claims 1-3 or the clustering method described in any one of claims 4-9.

16. A computer program product, comprising a computer program, which, when executed by a processor, implements the classification model training method according to any one of claims 1-3 or the clustering method according to any one of claims 4-9.

Images