Classification Processing Method, Apparatus, Computer Device, and Storage Medium

Abstract

The present application relates to a classification processing method, apparatus, computer device, and storage medium. The method includes: predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in an unlabeled data set based on a classification model trained using a labeled data set; labeling the unlabeled sample data whose predicted scores reach a filtering threshold to obtain a set of pseudo-labeled data; determining a new labeled data set based on the labeled data set and the set of pseudo-labeled data; after retraining the classification model using the new labeled data set, using the unlabeled data set after removing the sample data with the labeled predicted class labels as the new unlabeled data set, and returning to the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled data set based on the classification model trained using the labeled data set for iterative training until a training stop condition is met, to obtain a trained classification model. Using this method can improve classification accuracy.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and particularly to a classification processing method, apparatus, computer device, and storage medium. Background Art

[0002] In today's social life, classification models trained using machine learning methods are playing an increasingly important role. Supervised learning methods use labeled data sets to train classification models and cannot utilize unlabeled data. In order to utilize unlabeled data, semi-supervised learning methods have emerged.

[0003] In traditional semi-supervised learning methods, knowledge learned from labeled data is used to label unlabeled data, and then the labeled unlabeled data is used to train a classification model. However, only using the knowledge learned from labeled data to label unlabeled data cannot guarantee the accuracy of the labels, resulting in a decrease in the accuracy of the classification model. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a classification processing method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve accuracy.

[0005] In a first aspect, this application provides a classification processing method. The method includes:

[0006] Based on a classification model trained using a labeled data set, predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in an unlabeled data set;

[0007] For unlabeled sample data whose predicted scores reach a filtering threshold, labeling the corresponding predicted class labels to obtain a set of pseudo-labeled data; the pseudo-labeled data is sample data with the corresponding predicted class labels added;

[0008] Based on the labeled data set and the set of pseudo-labeled data, determining a new labeled data set;

[0009] After retraining the classification model using the new labeled data set, using the unlabeled data set after removing the sample data with the labeled predicted class labels as the new unlabeled data set, and returning to the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled data set based on the classification model trained using the labeled data set for iterative training until a training stop condition is met, to obtain a trained classification model.

[0010] In a second aspect, this application also provides a classification processing apparatus. The apparatus includes:

[0011] A prediction module, configured to predict the predicted class labels and corresponding predicted scores of each unlabeled sample data in an unlabeled dataset based on a classification model trained using a labeled dataset;

[0012] A determination module, configured to, for the unlabeled sample data whose predicted scores reach a filtering threshold, label the corresponding predicted class labels to obtain a set of pseudo-labeled data; the pseudo-labeled data is sample data with the corresponding predicted class labels added; based on the labeled dataset and the set of pseudo-labeled data, determine a new labeled dataset;

[0013] A training module, configured to, after retraining the classification model using the new labeled dataset, use the unlabeled dataset after removing the sample data with the labeled predicted class labels as the new unlabeled dataset, and return the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the classification model trained using the labeled dataset for iterative training until a training stop condition is met, to obtain a trained classification model.

[0014] In one embodiment, the determination module is further configured to, for each unlabeled sample data, if the predicted class label of the unlabeled sample data belongs to a head class label and the predicted score reaches a first filtering threshold, label the corresponding predicted class label for the unlabeled sample data;

[0015] if the predicted class label belongs to a tail class label and the predicted score reaches a second filtering threshold, label the corresponding predicted class label for the unlabeled sample data; the second filtering threshold is less than the first filtering threshold.

[0016] In one embodiment, the determination module is further configured to, based on the number of pseudo-labeled data under each predicted class label in the set of pseudo-labeled data, respectively determine the first weights corresponding to the pseudo-labeled data under each predicted class label; the magnitude of the first weight corresponding to the pseudo-labeled data under a predicted class label is negatively correlated with the number of pseudo-labeled data under the predicted class label; assign a second weight to the labeled data in the labeled dataset; generate a new labeled dataset based on the pseudo-labeled data with the first weights assigned and the labeled data with the second weights assigned.

[0017] In one embodiment, the set of pseudo-labeled data includes a plurality of predicted class labels; the determining module is further configured to sort each predicted class label in descending order of the number of pseudo-labeled data under each predicted class label among the plurality of predicted class labels; for each current predicted class label after sorting, calculate the ratio of the number of pseudo-labeled data under the associated label to the number of pseudo-labeled data under the reference class label according to the current weight, and determine the first weight of the pseudo-labeled data under the current predicted class label.

[0018] Wherein, the current weight calculation associated label is a predicted class label among the plurality of predicted class labels that is related to the weight calculation of the current predicted class label; the number of pseudo-labeled data under the current weight calculation associated label is greater than the number of pseudo-labeled data under the weight calculation associated label of the previous predicted class label and less than the number of pseudo-labeled data under the weight calculation associated label of the next predicted class label; the weight calculation associated labels of different predicted class labels are different; the reference class label is the predicted class label with the largest number of pseudo-labeled data among the plurality of predicted class labels.

[0019] In one embodiment, the training module is further configured to determine a weighted loss function of the classification model by using the first weight of the pseudo-labeled data and the second weight of the labeled data in the new labeled data set; and retrain the classification model with the new labeled data set in the direction of minimizing the loss value of the weighted loss function.

[0020] In one embodiment, the classification model is a text classification model; the prediction module is further configured to perform word segmentation on the labeled sample data in the labeled data set, and perform random uniform sampling on the sample data after word segmentation to obtain training sample data; and train the text classification model with the training sample data.

[0021] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0022] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0023] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0024] The above classification processing method, device, computer device, storage medium, and computer program product are based on a classification model trained using a labeled dataset to predict the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset; for the unlabeled sample data whose predicted scores reach the filtering threshold, mark the corresponding predicted class labels to obtain a set of pseudo-labeled data; the pseudo-labeled data is sample data with the corresponding predicted class labels added; based on the labeled dataset and the set of pseudo-labeled data, determine a new labeled dataset; after retraining the classification model using the new labeled dataset, use the unlabeled dataset after removing the sample data with the labeled predicted class labels as the new unlabeled dataset, and return to the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the classification model trained using the labeled dataset for iterative training until the training stop condition is met, obtaining a trained classification model. By continuously using the classification model to predict the unlabeled data, determining the set of pseudo-labeled data, then obtaining a new labeled dataset based on the labeled dataset and the set of pseudo-labeled data, and repeating the training of the classification model using the new labeled dataset, the effective utilization of the unlabeled data is achieved, and the knowledge learned from the labeled data and the unlabeled data can be used to label the unlabeled data and then train the classification model, improving the accuracy. Description of the Drawings

[0025] Figure 1 It is an application environment diagram of the classification processing method in an embodiment;

[0026] Figure 2 It is a flowchart of the classification processing method in an embodiment;

[0027] Figure 3 It is a schematic diagram of the principle of the classification processing method in an embodiment;

[0028] Figure 4 It is a structural block diagram of the classification processing device in an embodiment;

[0029] Figure 5 It is an internal structure diagram of a computer device in an embodiment;

[0030] Figure 6 It is an internal structure diagram of a computer device in another embodiment. Detailed Description of the Embodiments

[0031] In order to make the objectives, technical solutions, and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] The classification processing method provided by the embodiments of the present application can be applied to, for example, Figure 1 the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed in the cloud or other network servers. The server 104 can predict the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the classification model trained using the labeled dataset; the server 104 can mark the corresponding predicted class labels for the unlabeled sample data whose predicted scores reach the filtering threshold to obtain a set of pseudo-labeled data; the pseudo-labeled data is the sample data with the corresponding predicted class labels added; the server 104 can determine a new labeled dataset based on the labeled dataset and the set of pseudo-labeled data; after the server 104 retrains the classification model using the new labeled dataset, it can use the unlabeled dataset after removing the sample data with the labeled predicted class labels as the new unlabeled dataset, and return to the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the classification model trained using the labeled dataset for iterative training until the training stop condition is met, and obtain the trained classification model. It can be understood that the server 104 can send the trained classification model to the terminal 102, and the terminal 102 can load the trained classification model.

[0033] Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0034] In one embodiment, as Figure 2 shown, a classification processing method is provided. Taking the server in Figure 1 as an example for description, it can be understood that this method can also be applied to the terminal, and can also be applied to a system including the terminal and the server, and is implemented through the interaction between the terminal and the server. In this embodiment, the following steps are included:

[0035] Step 202, predict the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the classification model trained using the labeled dataset.

[0036] Among them, the predicted class label is the label obtained by the classification model predicting the unlabeled sample data. The prediction score is used to evaluate the quality of the predicted class label. It can be understood that the higher the quality of the predicted class label, the higher the corresponding prediction score. The labeled data set is a set of sample data labeled with class labels. The unlabeled data set is a set of sample data without labeled class labels.

[0037] Specifically, the server can use a tokenizer to preprocess the labeled data set and train a classification model using the preprocessed labeled data set. The server can predict the predicted class labels and corresponding prediction scores of each unlabeled sample data in the unlabeled data set through the classification model. It can be understood that the server can calculate the classification confidence of predicting the unlabeled sample data to obtain the prediction score. The higher the prediction score of the sample data, the higher the classification confidence of the sample data.

[0038] In one embodiment, the server can determine a tokenizer corresponding to the classification model. It can be understood that different classification models are adapted to different tokenizers. The tokenizer can include at least one of a pre-trained language representation model (BERT, Bidirectional Encoder Representation from Transformers) and a convolutional neural network (CNN, Convolutional Neural Networks), etc.

[0039] Step 204, for the unlabeled sample data whose prediction score reaches the filtering threshold, label the corresponding predicted class label to obtain a set of pseudo-labeled data; based on the labeled data set and the set of pseudo-labeled data, determine a new labeled data set.

[0040] Among them, the pseudo-labeled data is the sample data with the corresponding predicted class label added.

[0041] Specifically, the server can set a filtering threshold for the prediction score according to the number of sample data under each class label in the labeled data set. It can be understood that the number of sample data under each class label in the labeled data set may be unbalanced, and a higher filtering threshold is set for the class label with a larger number of sample data. The server can label the corresponding predicted class label for the unlabeled sample data whose prediction score reaches the filtering threshold to obtain a set of pseudo-labeled data. The server can integrate the labeled data set and the set of pseudo-labeled data to obtain a new labeled data set, that is, add the pseudo-labeled data to the labeled data set to obtain a new labeled data set.

[0042] Step 206, after retraining the classification model using the new labeled dataset, use the unlabeled dataset after removing the sample data with the predicted class labels that have been labeled as the new unlabeled dataset, and return to the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the classification model trained using the labeled dataset for iterative training until the training stop condition is met, and obtain the trained classification model.

[0043] Specifically, the server can retrain the classification model using the new labeled dataset, and use the unlabeled dataset after removing the sample data with the predicted class labels that have been labeled as the new unlabeled dataset, and return to the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the classification model trained using the labeled dataset for iterative training until at least one of the training stop conditions of model convergence and the unlabeled dataset being empty is met, and obtain the trained classification model.

[0044] In one embodiment, the server can select a validation set from the labeled dataset and count the prediction accuracy of the classification model for the sample data in the validation set to evaluate the convergence of the classification model by determining that the prediction accuracy no longer increases or increases insignificantly.

[0045] In the above classification processing method, based on the classification model trained using the labeled dataset, predict the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset; for the unlabeled sample data whose predicted scores reach the filtering threshold, label the corresponding predicted class labels to obtain a set of pseudo-labeled data; the pseudo-labeled data is the sample data with the corresponding predicted class labels added; based on the labeled dataset and the set of pseudo-labeled data, determine the new labeled dataset; after retraining the classification model using the new labeled dataset, use the unlabeled dataset after removing the sample data with the predicted class labels that have been labeled as the new unlabeled dataset, and return to the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the classification model trained using the labeled dataset for iterative training until the training stop condition is met, and obtain the trained classification model. By continuously using the classification model to predict the unlabeled data, determining the set of pseudo-labeled data, then obtaining the new labeled dataset based on the labeled dataset and the set of pseudo-labeled data, and repeating the training of the classification model using the new labeled dataset, the effective utilization of the unlabeled data is realized, and the knowledge learned from the labeled data and the unlabeled data can be used to label the unlabeled data and then train the classification model, improving the accuracy.

[0046] In one embodiment, for unlabeled sample data whose prediction scores reach the filtering threshold, labeling the corresponding prediction class labels includes: for each unlabeled sample data, if the prediction class label of the unlabeled sample data belongs to the head class label and the prediction score reaches the first filtering threshold, then label the corresponding prediction class label for the unlabeled sample data; if the prediction class label belongs to the tail class label and the prediction score reaches the second filtering threshold, then label the corresponding prediction class label for the unlabeled sample data; the second filtering threshold is less than the first filtering threshold.

[0047] Among them, the head class label is the class label with a large number of sample data in the labeled dataset. The tail class label is the class label with a small number of sample data in the labeled dataset.

[0048] Specifically, the server can determine the first filtering threshold set for the head class label and the second filtering threshold set for the tail class label. For each unlabeled sample data, determine the type of the prediction class label of the unlabeled sample data. If the prediction class label of the unlabeled sample data belongs to the type of the head class label, then compare the prediction score of the unlabeled sample data with the first filtering threshold. If the prediction score reaches the first filtering threshold, it is determined that the unlabeled sample data can be retained, and thus the corresponding prediction class label is labeled for the unlabeled sample data. If the prediction class label belongs to the type of the tail class label, then compare the prediction score of the unlabeled sample data with the second filtering threshold. If the prediction score reaches the second filtering threshold, it is determined that the unlabeled sample data can be retained, and thus the corresponding prediction class label is labeled for the unlabeled sample data.

[0049] In one embodiment, the server can determine the head class label and the tail class label according to the number of sample data under each class label in the labeled dataset. For example, the server can determine the division threshold according to the number of sample data under each class label in the labeled dataset. The class label with the number of sample data less than the division threshold belongs to the tail class label, and the class label with the number of sample data greater than the division threshold belongs to the head class label. It can be understood that the division threshold can be preset.

[0050] In one embodiment, the server can sort the class labels according to the number of sample data under each class label in the labeled dataset, and count the difference between the sample data under adjacent sorted class labels to determine the change in the number of sample data of adjacent sorted class labels, and use the adjacent class labels with the largest change in quantity as the division boundaries of the head class label and the tail class label respectively.

[0051] In this embodiment, different filtering thresholds are adopted according to the head category label and the tail category label to which the predicted category label belongs. It can be understood that if the number of sample data under each category label is unbalanced, the classification model trained has a high recall rate for the sample data under the head category label and a low recall rate for the sample data under the tail category label; however, it has a low precision rate for the sample data under the head category label and a high precision rate for the sample data under the tail category label. By using a large filtering threshold for the head category label and a small filtering threshold for the tail category label, the accuracy and quantity balance of the marked pseudo-labeled data are ensured, and a higher-quality new labeled data set can be obtained, thereby improving the performance of the trained classification model.

[0052] In one embodiment, based on the labeled data set and the set of pseudo-labeled data, determining the new labeled data set includes: respectively determining the first weight corresponding to the pseudo-labeled data under each predicted category label based on the number of pseudo-labeled data under each predicted category label in the set of pseudo-labeled data; the magnitude of the first weight corresponding to the pseudo-labeled data under the predicted category label is negatively correlated with the number of pseudo-labeled data under the predicted category label; assigning a second weight to the labeled data in the labeled data set; generating a new labeled data set based on the pseudo-labeled data after the first weight is assigned and the labeled data after the second weight is assigned.

[0053] Specifically, if the number of pseudo-labeled data under the predicted category label in the set of pseudo-labeled data is smaller, the server can assign a larger first weight to the pseudo-labeled data under the predicted category label. The value of the second weight can be greater than or equal to the maximum value of the first weight. The server can assign a second weight to the labeled data in the labeled data set and integrate the pseudo-labeled data after the first weight is assigned and the labeled data after the second weight is assigned to generate a new labeled data set. It can be understood that the sample data in the new labeled data set has weights of different magnitudes, and when the server uses the new labeled data set to train classification samples, it can preferentially adopt the sample data with a larger weight.

[0054] In this embodiment, the magnitude of the first weight corresponding to the pseudo-labeled data under the predicted category label is negatively correlated with the number of pseudo-labeled data under the predicted category label. By respectively assigning the first weight and the second weight to the pseudo-labeled data and the labeled data, the balance of the sample data in the new labeled data set is ensured. In addition, the pseudo-labeled data in the new labeled data set provides semantic diversity. Using the new labeled data set to train the classification model can mitigate the risk of overfitting or underfitting during training and further improve the performance of the trained classification model.

[0055] In one embodiment, a set of pseudo-labeled data includes multiple predicted class labels; determining first weights corresponding to the pseudo-labeled data under each predicted class label respectively based on the number of pseudo-labeled data under each predicted class label in the set of pseudo-labeled data includes: sorting each predicted class label in descending order according to the number of pseudo-labeled data under each predicted class label among the multiple predicted class labels; for each current predicted class label after sorting, determining the first weight of the pseudo-labeled data under the current predicted class label according to the ratio of the number of pseudo-labeled data under the associated label of the current weight calculation to the number of pseudo-labeled data under the reference class label; wherein, the associated label of the current weight calculation is a predicted class label related to the weight calculation of the current predicted class label among the multiple predicted class labels; the number of pseudo-labeled data under the associated label of the current weight calculation is greater than the number of pseudo-labeled data under the associated label of the previous predicted class label and less than the number of pseudo-labeled data under the associated label of the next predicted class label; the associated labels of different predicted class labels are different; the reference class label is the predicted class label with the largest number of pseudo-labeled data among the multiple predicted class labels.

[0056] Specifically, the server can sort each predicted class label in descending order according to the number of pseudo-labeled data under each predicted class label among the multiple predicted class labels. The server can use the predicted class label ranked first as the reference class label. The server can, for each current predicted class label after sorting, use the other predicted class label symmetric to the sorting of the current predicted class label as the associated label of the current weight calculation, and determine the first weight of the pseudo-labeled data under the current predicted class label according to the ratio of the number of pseudo-labeled data under the associated label of the current weight calculation to the number of pseudo-labeled data under the reference class label.

[0057] In one embodiment, the server can use the formula to assign a first weight of size μ l to the pseudo-labeled data under the predicted class label ranked l in the set of pseudo-labeled data. Wherein, L refers to the number of types of predicted class labels in this pseudo-labeled data set; α is the first weight exponent, α≥0; N L+1-l is the number of pseudo-labeled data under the predicted class label ranked L + 1 - l. It can be understood that the predicted class label ranked L + 1 - l is the associated label of the weight calculation of the predicted class label ranked l, and the predicted class label and the corresponding associated label of the weight calculation are symmetric in sorting.

[0058] In one embodiment, the server may determine a first weight index. For example, the server may determine the first weight index according to the current training round. The first weight index may be a preset fixed value. It can be understood that the maximum value of the first weight is 1, that is, the unit weight. The second weight may be the unit weight.

[0059] In one embodiment, the server may assign a first weight to the newly labeled pseudo-labeled data and a second weight to the existing pseudo-labeled data in the labeled dataset. It can be understood that the existing pseudo-labeled data in the labeled dataset is equivalent to labeled data for the training of the new round and no longer belongs to the set of pseudo-labeled data for the new round.

[0060] In this embodiment, by sorting each current predicted class label, for each current predicted class label after sorting, according to the current weight, calculating the ratio of the number of pseudo-labeled data under the associated label to the number of pseudo-labeled data under the reference class label, and determining the first weight of the pseudo-labeled data under the current predicted class label, the balance of the sample data in the new labeled dataset is ensured, and thus the performance of the trained classification model can be improved.

[0061] In one embodiment, the method further includes: determining a weighted loss function of the classification model by using the first weight of the pseudo-labeled data and the second weight of the labeled data in the new labeled dataset; and retraining the classification model using the new labeled dataset in the direction of minimizing the loss value of the weighted loss function.

[0062] Specifically, the server may use the first weight of the pseudo-labeled data and the second weight of the labeled data in the new labeled dataset as the loss weights of the loss function to calculate the weighted loss function of the classification model. The server may retrain the classification model using the new labeled dataset in the direction of minimizing the loss value of the weighted loss function.

[0063] In one embodiment, the number of sample data under each class label in the labeled dataset is unbalanced, and the server may adopt a weighted cross-entropy loss function. It can be understood that the weighted cross-entropy loss function can solve the problem of class label imbalance in the labeled dataset.

[0064] In this embodiment, by adopting a weighted loss function and retraining the classification model using the new labeled dataset in the direction of minimizing the loss value of the weighted loss function, the problem of class label imbalance can be well solved, and thus the performance of the trained classification model is improved.

[0065] In one embodiment, the classification model is a text classification model; before predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the classification model trained using the labeled dataset, it further includes: performing word segmentation on the labeled sample data in the labeled dataset, and performing random uniform sampling on the sample data after word segmentation to obtain training sample data; using the training sample data to train the text classification model.

[0066] Specifically, the server can perform word segmentation on the labeled sample data in the labeled dataset, and perform random uniform sampling on the sample data after word segmentation to obtain training sample data. The server can use the training sample data to train the text classification model. It can be understood that the server can also perform word segmentation on the sample data in the new labeled dataset, and perform random uniform sampling on the sample data after word segmentation to obtain new training sample data.

[0067] In one embodiment, the server can perform random uniform sampling according to the weight size of the sample data in the new labeled dataset. Among them, the greater the weight of the sample data, the greater the probability of being sampled.

[0068] In this embodiment, by performing word segmentation on the labeled sample data in the labeled dataset, and performing random uniform sampling on the sample data after word segmentation to obtain training sample data; using the training sample data to train the text classification model, so that pseudo-labeled data can be obtained based on the text training model, and then using the set of pseudo-labeled data and the new labeled dataset generated by the labeled dataset to retrain the text classification model, improving the performance of the text classification model.

[0069] In addition, for text classification tasks, there is often a problem of unbalanced sample data. Although resampling can be used to change the distribution of sample data in training, or reweighting can be used to change the loss function to improve the precision and recall rate of text classification. However, resampling or reweighting will lead to overfitting in the tail classes and underfitting in the head classes. And the pseudo-labeled data in the new labeled dataset provides semantic diversity, alleviating the problems of overfitting in the tail classes and underfitting in the head classes, and further improving the performance of the text classification model.

[0070] In one embodiment, as Figure 3The principle schematic diagram of the classification processing method is provided as shown. The server can preprocess the original data set to determine the labeled data set and the unlabeled data set. The server can use the labeled data set to train a classification model, and predict the predicted class labels of the unlabeled sample data in the unlabeled data set through the classification model. The server can label the predicted class labels of the unlabeled sample data to obtain pseudo-labeled data, and then add the pseudo-labeled data to the labeled data set to obtain a new labeled data set. It can be understood that the sample data in the unlabeled data set corresponding to the pseudo-labeled data will be screened out. The server can use the new labeled data set to train the classification model. If the training stop condition is met, that is, the model converges or the unlabeled data set is empty, the trained classification model is obtained; otherwise, return to the step of predicting the predicted class labels of the unlabeled sample data in the unlabeled data set through the classification model until the training stop condition is met.

[0071] In this embodiment, by learning the knowledge of the labeled sample data and the unlabeled sample data in the data set, it is applicable to any classification task and data set, does not depend on the architecture, and has good generality.

[0072] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps in other steps.

[0073] Based on the same inventive concept, an embodiment of the present application also provides a classification processing device for implementing the above-mentioned classification processing method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the classification processing device provided below can refer to the limitations on the classification processing method in the above text, and will not be repeated here.

[0074] In one embodiment, as Figure 4 shown, a classification processing device 400 is provided, including: a prediction module 402, a determination module 404, and a training module 406, where:

[0075] A prediction module 402, configured to predict the predicted class labels and corresponding predicted scores of each unlabeled sample data in an unlabeled dataset based on a classification model trained using a labeled dataset;

[0076] A determination module 404, configured to, for the unlabeled sample data whose predicted scores reach a filtering threshold, label the corresponding predicted class labels to obtain a set of pseudo-labeled data; the pseudo-labeled data is the sample data with the corresponding predicted class labels added; determine a new labeled dataset based on the labeled dataset and the set of pseudo-labeled data;

[0077] A training module 406, configured to, after retraining the classification model using the new labeled dataset, use the unlabeled dataset after removing the sample data with the labeled predicted class labels as the new unlabeled dataset, and return the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the classification model trained using the labeled dataset for iterative training until a training stop condition is met, to obtain a trained classification model.

[0078] In one embodiment, the determination module 404 is further configured to, for each unlabeled sample data, if the predicted class label of the unlabeled sample data belongs to a head class label and the predicted score reaches a first filtering threshold, label the corresponding predicted class label for the unlabeled sample data;

[0079] If the predicted class label belongs to a tail class label and the predicted score reaches a second filtering threshold, label the corresponding predicted class label for the unlabeled sample data; the second filtering threshold is less than the first filtering threshold.

[0080] In one embodiment, the determination module 404 is further configured to, based on the number of pseudo-labeled data under each predicted class label in the set of pseudo-labeled data, respectively determine the first weight corresponding to the pseudo-labeled data under each predicted class label; the magnitude of the first weight corresponding to the pseudo-labeled data under a predicted class label is negatively correlated with the number of pseudo-labeled data under the predicted class label; assign a second weight to the labeled data in the labeled dataset; generate a new labeled dataset based on the pseudo-labeled data with the first weight assigned and the labeled data with the second weight assigned.

[0081] In one embodiment, the set of pseudo-labeled data includes multiple predicted class labels; the determination module 404 is further configured to sort each predicted class label in descending order according to the number of pseudo-labeled data under each predicted class label among the multiple predicted class labels; for each current predicted class label after sorting, calculate the ratio of the number of pseudo-labeled data under the associated label to the number of pseudo-labeled data under the reference class label according to the current weight, and determine the first weight of the pseudo-labeled data under the current predicted class label;

[0082] Among them, the current weight calculation associated label is the prediction category label related to the weight calculation of the current prediction category label among multiple prediction category labels; the number of pseudo-labeled data under the current weight calculation associated label is larger than the number of pseudo-labeled data under the weight calculation associated label of the previous prediction category label and smaller than the number of pseudo-labeled data under the weight calculation associated label of the next prediction category label; the weight calculation associated labels of different prediction category labels are different; the reference category label is the prediction category label with the largest number of pseudo-labeled data among multiple prediction category labels.

[0083] In one embodiment, the training module 406 is further configured to determine a weighted loss function of the classification model by using the first weight of the pseudo-labeled data and the second weight of the labeled data in the new labeled data set; and retrain the classification model by using the new labeled data set in the direction of minimizing the loss value of the weighted loss function.

[0084] In one embodiment, the classification model is a text classification model; the prediction module 402 is further configured to perform word segmentation on the labeled sample data in the labeled data set, and perform random uniform sampling on the sample data after word segmentation to obtain training sample data; and use the training sample data to train the text classification model.

[0085] Each module in the above classification processing device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0086] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the training data of the classification model. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a classification processing method.

[0087] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structural diagram may be as shown in Figure 6 . The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a classification processing method. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen may be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0088] Those skilled in the art can understand that Figure 5 and Figure 6 the structures shown in are only block diagrams of some structures related to the solution of the present application, and do not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0089] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, it implements the steps in the above method embodiments.

[0090] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0091] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0092] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0093] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., and are not limited thereto.

[0094] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0095] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A classification processing method, executed by a computer device, characterized in that, The method includes: Performing word segmentation on the labeled sample data in the labeled dataset, and performing random uniform sampling on the sample data after word segmentation to obtain training sample data, where the labeled dataset is a set of sample data labeled with class labels; Training a text classification model using the training sample data; Based on the text classification model trained using the training sample data, predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset, where the unlabeled dataset is a set of sample data without labeled class labels; For the unlabeled sample data whose predicted scores reach the filtering threshold, labeling the corresponding predicted class labels to obtain a set of pseudo-labeled data; the pseudo-labeled data is sample data with the corresponding predicted class labels added; Based on the labeled dataset and the set of pseudo-labeled data, determining a new labeled dataset; Performing word segmentation on the sample data in the new labeled dataset, and performing random uniform sampling on the sample data after word segmentation to obtain new training sample data; After retraining the text classification model using the new training sample data, taking the unlabeled dataset after removing the sample data with the labeled predicted class labels as the new unlabeled dataset, and returning to the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the text classification model trained using the training sample data for iterative training until the training stop condition is met, to obtain a trained text classification model.

2. The method according to claim 1, characterized in that, The labeling the corresponding predicted class labels for the unlabeled sample data whose predicted scores reach the filtering threshold includes: For each unlabeled sample data, if the predicted class label of the unlabeled sample data belongs to the head class label and the predicted score reaches the first filtering threshold, then labeling the corresponding predicted class label for the unlabeled sample data; If the predicted class label belongs to the tail class label and the predicted score reaches the second filtering threshold, then labeling the corresponding predicted class label for the unlabeled sample data; the second filtering threshold is less than the first filtering threshold.

3. The method according to claim 1, wherein The determining a new labeled dataset based on the labeled dataset and the set of pseudo-labeled data includes: Based on the number of pseudo-labeled data under each predicted class label in the set of pseudo-labeled data, respectively determining the first weights corresponding to the pseudo-labeled data under each predicted class label; the magnitude of the first weight corresponding to the pseudo-labeled data under a predicted class label is negatively correlated with the number of pseudo-labeled data under the predicted class label; Assigning a second weight to the labeled data in the labeled dataset; Generating a new labeled dataset based on the pseudo-labeled data after assigning the first weights and the labeled data after assigning the second weights.

4. The method according to claim 3, wherein The set of pseudo-labeled data includes multiple predicted class labels; the respectively determining the first weights corresponding to the pseudo-labeled data under each predicted class label in the set of pseudo-labeled data includes: Sort each prediction class label in descending order according to the number of pseudo-labeled data under each prediction class label among the multiple prediction class labels; For each current prediction class label after sorting, calculate the ratio of the number of pseudo-labeled data under the associated label to the number of pseudo-labeled data under the reference class label according to the current weight, and determine the first weight of the pseudo-labeled data under the current prediction class label; Wherein, the current weight calculation associated label is the prediction class label related to the weight calculation of the current prediction class label among the multiple prediction class labels; the number of pseudo-labeled data under the current weight calculation associated label is larger than the number of pseudo-labeled data under the weight calculation associated label of the previous prediction class label and smaller than the number of pseudo-labeled data under the weight calculation associated label of the next prediction class label; the weight calculation associated labels of different prediction class labels are different; the reference class label is the prediction class label with the largest number of pseudo-labeled data among the multiple prediction class labels.

5. The method according to claim 3, wherein The method further includes: Determine the weighted loss function of the text classification model by using the first weight of the pseudo-labeled data and the second weight of the labeled data in the new labeled data set; Re-train the text classification model with the new labeled data set in the direction of minimizing the loss value of the weighted loss function.

6. The method according to claim 2, characterized in that, The method further includes: Determine a division threshold according to the number of sample data under each category label in the labeled data set; Among them, the category label with the number of sample data less than the division threshold belongs to the tail category label, and the category label with the number of sample data greater than the division threshold belongs to the head category label.

7. A classification processing device, characterized in that, The device includes: A training module for performing word segmentation on the labeled sample data in the labeled data set, and performing random uniform sampling on the sample data after word segmentation to obtain training sample data, where the labeled data set is a set of sample data labeled with category labels; training the text classification model with the training sample data; A prediction module for predicting the prediction class label and the corresponding prediction score of each unlabeled sample data in the unlabeled data set based on the text classification model trained with the training sample data, where the unlabeled data set is a set of sample data without labeled category labels; A determination module for labeling the corresponding prediction class label for the unlabeled sample data whose prediction score reaches the filtering threshold to obtain a set of pseudo-labeled data; the pseudo-labeled data is the sample data with the corresponding prediction class label added; determining a new labeled data set based on the labeled data set and the set of pseudo-labeled data; The training module is further configured to perform word segmentation on the sample data in the new labeled data set, and perform random uniform sampling on the sample data after word segmentation to obtain new training sample data; The training module is further configured to, after retraining the text classification model using the new training sample data, use the unlabeled dataset obtained by removing the sample data with the labeled predicted class labels as the new unlabeled dataset, and return the step of predicting the predicted class labels and corresponding predicted scores of each unlabeled sample data in the unlabeled dataset based on the text classification model trained using the training sample data for iterative training until the training stop condition is met, thereby obtaining the trained text classification model.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When this computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

Images