Text classification model training method, text classification method and related equipment

Abstract

The invention discloses a training method of a text classification model, a text classification method and related equipment, which are used for improving the execution effect of a text classification task on the premise of remarkably reducing the amount of annotated data required in the text classification task. The text classification model training method comprises the steps of adjusting parameters of a first text classification model based on a labeled first text and a label of the first text in training data to obtain a second text classification model; classifying unlabeled second texts in the training data based on the second text classification model to obtain labels and confidence coefficients of the second texts; determining a fourth text from the second text based on the label and the confidence coefficient of the second text and a label of a third text similar to the second text in the training data; and adjusting parameters of the second text classification model based on the label of the first text and the label of the fourth text to obtain a third text classification model.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a training method for a text classification model, a text classification method, and related equipment. Background Art

[0002] In supervised learning, the lack of labeled data is the main obstacle to practical application, because it is difficult to obtain labeled data, and sometimes even requires domain experts to label them. Labeled data is even more scarce in text classification tasks.

[0003] How to improve the performance of text classification tasks while minimizing the amount of labeled data required in text classification tasks has become a hot research topic. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a text classification model training method, a text classification method and related equipment, which are used to improve the performance of text classification tasks while significantly reducing the amount of labeled data required in the text classification tasks.

[0005] In order to achieve the above objectives, the present application embodiment adopts the following technical solutions: In a first aspect, an embodiment of the present application provides a method for training a text classification model, comprising: Based on the annotated first text in the training data and the label of the first text, adjusting the parameters of the first text classification model to obtain a second text classification model; Classifying the unlabeled second text in the training data based on the second text classification model to obtain a label and a confidence level of the second text; Determine a fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data; Based on the label of the first text and the label of the fourth text, the parameters of the second text classification model are adjusted to obtain a third text classification model.

[0006] In a second aspect, an embodiment of the present application provides a text classification method, including: Get the target text; The target text is classified based on a third text classification model to obtain a predicted category of the target text; wherein the third text classification model is trained based on the training method of the text classification model provided in the first aspect.

[0007] In a third aspect, an embodiment of the present application provides a training device for a text classification model, comprising: A first adjustment module, configured to adjust parameters of the first text classification model based on the annotated first text in the training data and the label of the first text, so as to obtain a second text classification model; A first classification module, used for classifying the unlabeled second text in the training data based on the second text classification model to obtain a label and a confidence level of the second text; A determination module, configured to determine a fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data; The second adjustment module is used to adjust the parameters of the second text classification model based on the label of the first text and the label of the fourth text to obtain a third text classification model.

[0008] In a fourth aspect, an embodiment of the present application provides a text classification device, including: An acquisition module is used to acquire the target text; The second classification module is used to classify the target text based on a third text classification model to obtain a predicted category of the target text; wherein the third text classification model is trained based on the training method of the text classification model provided in the first aspect.

[0009] In a fifth aspect, an embodiment of the present application provides an electronic device, including: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the text classification model training method provided in the first aspect; or, the processor is configured to execute the instructions to implement the text classification method provided in the second aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the training method of a text classification model provided in the first aspect; or, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the text classification method provided in the second aspect.

[0011] In the seventh aspect, an embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps in the training method of the text classification model provided in the first aspect or the text classification method provided in the second aspect.

[0012] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: First, the parameters of the first classification model are adjusted using the labeled first text in the training data, and the performance of the obtained second text classification model is improved compared with the first text classification model; on this basis, a self-training method is introduced, that is, the second text classification model classifies the unlabeled second text in the training data to generate corresponding pseudo labels for the unlabeled second text, and selects the fourth text and its pseudo labels with higher pseudo-label accuracy, and combines the first text and its labels in the training data to adjust the parameters of the second text classification model to obtain a third text classification model.

[0013] In this way, with the help of limited annotated text and computing resources, the number of texts and their labels in the training data used for model training can be gradually increased, the training data can be enhanced, thereby improving the generalization ability of the model, and obtaining a third text classification model with high accuracy and high generalization ability, and the amount of annotated data required in the model training process can be minimized.

[0014] In addition, the selection of the fourth text and its pseudo-label is based on the pseudo-label and confidence of the second text and the label of the third text similar to the second text in the training data. This not only reduces the amount of calculation and is efficient, but also ensures that the selected fourth text has high confidence and consistency, reduces the introduction of label noise, and further improves the accuracy and generalization ability of the third text classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flowchart of a method for training a text classification model provided for one embodiment of the present application; Figure 2 A flowchart of a method for training a text classification model provided in another embodiment of the present application; Figure 3 A flowchart of a method for training a text classification model provided in yet another embodiment of the present application; Figure 4 A flowchart of a text classification method provided for an embodiment of the present application; Figure 5 A schematic diagram of the structure of a text classification model training device provided in one embodiment of the present application; Figure 6 A schematic diagram of the structure of a text classification device provided for one embodiment of the present application; Figure 7 A schematic diagram of the structure of an electronic device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0017] The terms "first", "second", etc. in this specification and claims are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in this specification and claims means at least one of the connected objects, and the character " / " generally means that the objects associated with each other are in an "or" relationship.

[0018] Explanation of relevant terms: Active Learning: An iterative labeling process used to train a model by obtaining a small labeled subset in the absence of labeled data. The core idea is to select the most informative data for the model to be labeled in each iteration, thereby minimizing the amount of labeled data required.

[0019] Self-Training: A semi-supervised learning method that generates pseudo labels from unlabeled data and uses these pseudo labels to enhance training data, thereby reducing the need for labeled data. The self-training method can be combined with active learning to use the model's prediction results to pseudo-label unlabeled data and add these pseudo-labeled data to the training data.

[0020] Transformer model: A deep learning model architecture based on the attention mechanism, widely used in natural language processing tasks. Its main feature is that it can handle long-distance dependencies and improve the effect of text representation. Representative models include BERT and GPT.

[0021] BERT (Bidirectional Encoder Representations from Transformers): A pre-trained language model that is trained on a large corpus using a bidirectional Transformer architecture and can generate high-quality text representations for a variety of natural language processing tasks.

[0022] Pseudo-label: A label obtained by predicting unlabeled data through a model. These labels are used to enhance training data and improve the generalization ability of the model.

[0023] K-Nearest Neighbors (KNN): An instance-based classification algorithm that calculates the distance between a new instance and known instances in the training data, finds the K nearest neighbors, and makes predictions based on the labels of these neighbors.

[0024] Contrastive Representation Learning: A method that trains by contrasting pairs of examples so that semantically similar inputs are closer in the embedding space and dissimilar inputs are farther apart, thereby learning semantically rich representations.

[0025] Siamese Network: A neural network structure that contains two or more sub-networks that share weights. It is usually used for contrastive learning tasks and is trained by comparing the similarities between input pairs.

[0026] In order to improve the performance of text classification tasks while minimizing the amount of labeled data required in text classification tasks, an embodiment of the present application proposes a training method for a text classification model. First, the parameters of the first classification model are adjusted using the labeled first text in the training data, and the performance of the obtained second text classification model is improved compared with the first text classification model. On this basis, a self-training method is introduced, that is, the second text classification model classifies the unlabeled second text in the training data to generate corresponding pseudo-labels for the unlabeled second text, and selects a fourth text and its pseudo-label with higher pseudo-label accuracy. Combined with the first text and its label in the training data, the parameters of the second text classification model are adjusted to obtain a third text classification model.

[0027] In this way, with the help of limited annotated text and computing resources, the number of texts and their labels in the training data used for model training can be gradually increased, the training data can be enhanced, thereby improving the generalization ability of the model, and obtaining a third text classification model with high accuracy and high generalization ability, and the amount of annotated data required in the model training process can be minimized.

[0028] In addition, the selection of the fourth text and its pseudo-label is based on the pseudo-label and confidence of the second text and the label of the third text similar to the second text in the training data. This not only reduces the amount of calculation and is efficient, but also ensures that the selected fourth text has high confidence and consistency, reduces the introduction of label noise, and further improves the accuracy and generalization ability of the third text classification model.

[0029] Based on the third text classification model obtained by the above training method, an embodiment of the present application also provides a text classification method, which uses the third text classification model obtained by the above training method to classify the target text, and the obtained prediction category has high reliability and accuracy.

[0030] It should be understood that the training method of the text classification model and the text classification method provided in the embodiments of the present application can be performed by an electronic device. The so-called electronic device here can include terminal devices, such as smart phones, tablet computers, laptops, desktop computers, intelligent voice interaction devices, smart home appliances, smart watches, vehicle terminals, aircraft, etc.; or, the electronic device can also include a server, such as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0031] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0032] Please refer to Figure 1 , is a flowchart of a method for training a text classification model provided by an embodiment of the present application, the method comprising the following steps: S102: Based on the annotated first text in the training data and the label of the first text, adjust the parameters of the first text classification model to obtain a second text classification model.

[0033] The training data can be divided into a labeled data set and an unlabeled data set. The text in the labeled data set is labeled with a label indicating a category, and the text in the unlabeled data set is not labeled. The first text is the text in the labeled data set. In practical applications, all the text in the labeled data set can be used as the first text, or some text can be randomly selected from the labeled data set as the first text.

[0034] The first text classification model refers to the text classification model that needs to be trained. The label of the first text can provide a supervisory signal for the first text classification model. The first text and its label are input into the first text classification model, and the first text classification model classifies the first text to obtain the predicted category of the first text; then, based on the difference between the predicted category and the label of the first text, the loss of the first text classification model is determined, and then the parameters of the first text classification model are adjusted based on the loss, such as using the back propagation algorithm, the gradient descent algorithm, etc. to adjust the parameters of the first text classification model, thereby completing the preliminary training of the first text classification model and obtaining the second text classification model. The performance of the second text classification model is improved compared with the first text classification model.

[0035] S104, classifying the unlabeled second text in the training data based on the second text classification model to obtain a label and a confidence level of the second text.

[0036] The second text is the text in the unlabeled data set. In practical applications, all the text in the unlabeled data set can be used as the second text, or some text can be randomly selected from the unlabeled data set as the second text.

[0037] For each second text, the second text is input into the second text classification model for classification, and the label and confidence of the second text can be obtained. The label of the second text represents the category of the second text, and the confidence of the second text represents the probability that the category is the true category of the second text.

[0038] The label of the second text indicates the category of the second text, which may be one of a plurality of preset categories. For example, the plurality of categories include two categories, positive sentiment and negative sentiment. The second text is input into the second text classification model to obtain the probability that the category of the second text is positive sentiment and the probability that the category of the second text is negative sentiment; then, the category corresponding to the maximum probability is selected to obtain the label of the second text, and the probability corresponding to the category is used as the confidence of the second text.

[0039] It is worth noting that since the label of the second text is not obtained through annotation (such as manual annotation), but is learned by the second text classification model from the information contained in the second text, the label is also called a pseudo label. The confidence of the second text represents the probability that the category represented by the label of the second text is the true category of the second text.

[0040] S106 , determining a fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data.

[0041] The fourth text refers to the second text with high label quality. Considering that similar texts usually have the same category, based on the label of the second text and the label of the third text, as well as the confidence of the second text, it can be determined whether the label of the second text is accurate, and the second text with high-accuracy labels is selected as the fourth text in turn, as the data for subsequent training of the second text classification model, so as to expand the number of texts with labels and improve the accuracy and generalization ability of the second text classification model.

[0042] In one embodiment, the above S106 may include the following step: if the number of third texts corresponding to the same label as the second text is greater than a first number threshold, and the confidence of the second text is greater than the first confidence threshold, the second text is determined as the fourth text.

[0043] Among them, the first quantity threshold and the first confidence threshold can be determined according to actual needs. The first quantity threshold can be a preset value, or a value determined according to the number of third texts, such as setting the first quantity threshold to half the number of third texts according to the voting majority principle. The first confidence threshold can be a preset value, or it can be determined according to the confidence of all second texts. If the confidence of most second texts is between 0.4 and 0.7, the first confidence threshold is determined to be 0.5.

[0044] For example, for each second text, if the number of third texts similar to the second text is 100, the first quantity threshold is determined to be 50. Assuming that the first confidence threshold is 0.5, the labels of 60 second texts are the same as the label of the second text, and the confidence of the second text is 0.6, the second text is determined to be the fourth text.

[0045] In the above implementation, the fourth text is determined from the second text in combination with the confidence of the second text and the number of third texts with the same label as the second text. This not only ensures that the label of the fourth text has high confidence and consistency and reduces the introduction of label noise, but also enables the selection of the fourth text with a high-quality label under a looser first confidence threshold, thereby avoiding the problem of overly conservative fourth text selection caused by a strict first confidence threshold. This approach is more flexible in practical applications and can adapt to different training data and application scenarios.

[0046] In another embodiment, if the confidence of the second text is less than or equal to the first confidence threshold, but the label of the third text is the same as the label of the second text, the second text is determined as the fourth text. In this way, the problem of overly conservative selection of the fourth text caused by the strict first confidence threshold can be further avoided, and the number of texts used to train the second text model can be increased.

[0047] The embodiment of the present application shows some implementation methods of the above S106. Of course, it should be understood that the above S106 can also be implemented in other ways.

[0048] In the above S106, the third text in the training data that is similar to the second text can be obtained through various appropriate methods, which is not limited in this embodiment of the present application.

[0049] In one embodiment, the third text is determined by: calculating the distance between the second text and the text with a label in the training data; then selecting multiple texts that are closest to the second text as the third text similar to the second text. The distance between any two texts can be calculated in various ways, such as but not limited to one or more of the following ways: cosine similarity, edit distance, TF-IDF (term frequency-inverse document frequency) similarity, etc.

[0050] It is worth noting that there are multiple second texts. For each second text, the above implementation is used to determine a third text similar to the second text from the training data.

[0051] In another embodiment, the third text is determined by: determining the distance between the second text and the text with a label in the training data; clustering the second text and the text with a label in the training data based on the distance to obtain multiple clusters, each cluster containing at least one text; and selecting multiple texts from the cluster where the second text is located as the third text similar to the second text.

[0052] The texts with labels in the training data include: the annotated first texts and all the second texts with labels.

[0053] The second text and the text with labels in the training data are clustered by various clustering algorithms commonly used in the art, including but not limited to one or more combinations of the following algorithms: KNN algorithm, K-means algorithm, random forest algorithm, etc. In a particularly preferred solution, the KNN algorithm can be used to reduce the amount of calculation and improve the clustering efficiency while ensuring accuracy.

[0054] In practical applications, the second text and the labeled text in the training data can be mapped into low-dimensional vectors, such as BERT or other Transformer models, Siamese and other pre-trained language models, to embed these texts and obtain vectors of each text, which form an embedding space. By clustering the vectors in the embedding space, multiple clusters are obtained, each cluster includes at least one text vector; then, for each second text, multiple vectors closest to the vector are selected from the cluster where the vector of the second text is located, and the text represented by each selected vector is determined as a third text similar to the second text.

[0055] In the above implementation, the third text similar to the second text is determined by clustering, which is more efficient and accurate. In addition, clustering the second text and the text with labels in the training data can, on the one hand, provide richer context information, which helps to more accurately find the third text similar to the second text. On the other hand, the first text has an accurate label, which can be used to verify whether the label of the second text is accurate. The accuracy of the label of the fourth text is ensured through the above-mentioned screening mechanism for the fourth text.

[0056] S108: Based on the label of the first text and the label of the fourth text, adjust the parameters of the second text classification model to obtain a third text classification model.

[0057] Since the label of the fourth text has a high accuracy, the fourth text can be regarded as a new annotated text and used together with the first text to adjust the parameters of the second text classification model. In this way, with the help of limited annotated text and computing resources, the number of texts and their labels used for model training in the training data can be gradually increased, the training data can be enhanced, and the generalization ability of the model can be improved, and a third text classification model with high accuracy and high generalization ability can be obtained, and the amount of annotated data required in the model training process can be minimized.

[0058] In one embodiment, the above S108 includes the following steps: S1082a, classifying the first text based on the second text classification model to obtain a predicted category of the first text, and classifying the fourth text to obtain a predicted category of the fourth text.

[0059] S1084a, determining a first loss based on the label of the first text and the predicted category of the first text.

[0060] The first loss represents the loss generated by the second text classification model in classifying the first text. Specifically, for each first text, the difference between the label and the predicted category of the first text is calculated to obtain the loss of the second text; then, the losses of all first texts are summed to obtain the first loss.

[0061] The difference between the label of the first text and the predicted category may be represented in various appropriate ways, such as MSE (Mean Squared Error) or cross entropy, etc., which is not limited in the embodiment of the present application.

[0062] S1086a, determining a second loss based on the label of the fourth text and the predicted category of the fourth text.

[0063] The second loss represents the loss generated by the second text classification model in classifying the fourth text. Specifically, for each fourth text, the difference between the label and the predicted category of the fourth text is calculated to obtain the loss of the fourth text; then, the losses of all fourth texts are summed to obtain the second loss.

[0064] Among them, the difference between the label of the fourth text and the predicted category can be expressed in various appropriate ways, such as MSE or cross entropy, etc., which is not limited in this embodiment of the present application.

[0065] S1088a: Based on the first loss and the second loss, adjust the parameters of the second text classification model to obtain a third text classification model.

[0066] Specifically, the first loss and the second loss are added to obtain the loss of the second text classification model; then, based on the loss, the back propagation algorithm, the gradient descent algorithm, etc. are used to adjust the parameters of the second text classification model to obtain the third text classification model.

[0067] The above implementation takes into account the influence of the first text and the fourth text on the second text classification model, and the obtained third text classification model has higher accuracy and generalization ability.

[0068] In another embodiment, the above S108 includes the following steps: S1082b, based on the second text classification model, classify the first text to obtain a predicted category of the first text, and classify the fourth text to obtain a predicted category of the fourth text.

[0069] The specific implementation of the above S1082b is similar to the specific implementation of the above S1082a, and will not be repeated.

[0070] S1084b, determining a first loss based on the label of the first text and the predicted category of the first text.

[0071] The specific implementation of the above S1084b is similar to the specific implementation of the above S1084a, and will not be repeated.

[0072] S1086b, determining a second loss based on the label of the fourth text, the predicted category of the fourth text, and the weight of the fourth text.

[0073] Each fourth text has a corresponding weight, and the weight is used to adjust the proportion of the loss of the fourth text in the second loss.

[0074] Specifically, for each fourth text, the difference between the label and the predicted category of the fourth text is calculated to obtain the loss of the fourth text; then, based on the weight of each fourth text, the losses of all fourth texts are weighted summed to obtain the second loss.

[0075] S1088b: Based on the first loss and the second loss, adjust the parameters of the second text classification model to obtain a third text classification model.

[0076] The specific implementation of the above S1088b is similar to the specific implementation of the above S1088a, and will not be repeated.

[0077] Considering that the confidence of the label of the fourth text is usually lower than that of the label of the first text, if the fourth text and its label are directly used to train the second text classification model, the performance of the second text classification model will be affected. To this end, through the above implementation, when calculating the second loss, an appropriate weight is given to each fourth text, which can reduce the negative impact of the label of the fourth text on the second classification model, thereby improving the performance of the second classification model.

[0078] In the above implementation, the weight of each fourth text may be obtained in various appropriate ways, which is not limited in this embodiment of the present application.

[0079] In one implementation, the weight of the fourth text may be a numerical value less than 1 that is preset based on experience to reduce the proportion of the second loss, thereby reducing the impact of the fourth text on the performance of the second text classification model.

[0080] In another embodiment, the label of the fourth text represents one of the preset multiple categories. In this case, the weight of the fourth text is determined by: Step A1: determining the number of second texts in each category based on the labels of the second texts.

[0081] Step A2, determining the weight of each category based on the number of second texts under each category.

[0082] As an example, for each category, a ratio between the number of second texts in the category and the number of all second texts is determined as the weight of the category.

[0083] As another example, a first ratio between the number of all second texts and the number of multiple categories is determined; for each category, a weight of the category is determined based on a difference between the first ratio and the number of second texts under the category.

[0084] Since the first ratio reflects the mean number of second texts under each category, the difference between the first ratio and the number of second texts under each category can reflect the distribution difference of the second texts under each category. Determining the weight of each category based on the difference helps to adjust category imbalance.

[0085] For example, for each category, the difference between the first ratio and the number of second texts in the category is determined as the weight of the category. In this case, if the difference is larger, the weight of the category is larger; if the difference is smaller, the weight of the category is smaller. In this way, the impact of unbalanced categories on the second text classification model can be reduced, and the training effect of the second text classification model can be improved.

[0086] For another example, for each category, determine the maximum value between the second text quantity under the category and the second quantity threshold; determine the second ratio between the first ratio and the maximum value; and determine the second ratio as the weight of the category. Specifically, the second ratio is shown in the following formula (1): (1) in, represents the second ratio, Indicates the number of all second texts, Indicates the number of multiple categories, represents the number of second texts under the cth category, Indicates taking the maximum value, and the second quantity threshold is 1.

[0087] For another example, for each category, determine the maximum value between the second text quantity under the category and the second quantity threshold; determine the second ratio between the first ratio and the maximum value; map the second ratio to a preset value range to obtain the weight of the category.

[0088] The second ratio may be mapped in various appropriate ways, which are not limited in the present application. In a preferred solution, a sigmoid function may be used to map the second ratio.

[0089] The weight of each category can be determined by the following formula (2): (2) in, represents the weight of the cth category, represents the second ratio, and the second ratio can be determined by the above formula (1).

[0090] By mapping the second ratio to a numerical value as the weight of the corresponding category, compared with the above two methods, the weight will not be too large or too small, thereby effectively avoiding excessive changes in the weight and affecting the stability and training effect of the second text classification model.

[0091] Step A3: determining the weight of the fourth text based on the weight of the category represented by the label of the fourth text.

[0092] For example, for each fourth text, the weight of the category represented by the label of the fourth text is determined as the weight of the fourth text.

[0093] For another example, the parameters of the second text classification model include the weight ratio of the fourth text. In this case, the product of the weight and ratio of the category represented by the label of the fourth text is determined as the weight of the fourth text. Specifically, the weight of each fourth text can be determined by the following formula (3): (3) in, represents the weight of the ith fourth text, Indicates the category represented by the label of the ith fourth text The weight of Indicates the weight ratio of the fourth text.

[0094] In this way, the weight of the fourth text can be continuously adjusted during the model training process, thereby achieving the effect of controlling the influence of the fourth text label on the model training effect. For example, when the weight of the fourth text is relatively small, it means that the weight of the fourth text in the training is low, thereby reducing its potential negative impact.

[0095] Considering that the process of selecting the fourth text from the second text to train the second text classification model may cause the problem of unbalanced category distribution, that is, the number of second texts under different categories differs too much, which will seriously affect the training effect of the second text classification model. To this end, in the above implementation, the weight of each category is determined based on the number of second texts belonging to each category, and the weight of the category represented by the label of the second text is determined based on the weight of the category represented by the label of the fourth text. This can effectively adjust the category imbalance, thereby improving the training effect of the second text classification model and obtaining a third text classification model with higher classification accuracy.

[0096] It is worth noting that the above S102 to S108 is only a model training process. In practical applications, the above S102 to S108 can be performed multiple times to achieve multiple rounds of model training until a third text classification model with performance that meets the requirements is obtained.

[0097] During the first round of training, the first text in S102 is the manually annotated text selected from the annotated data set, and after the first round of training, the fourth text is added to the annotated data set as the new first text, and the fourth text is deleted from the unannotated data set; in each subsequent round of training, the first text in S102 is the text selected from the annotated data set, which can be the manually annotated text or the fourth text newly added to the annotated data set after the previous round of training.

[0098] In addition, after each round of training, the model performance is evaluated using a validation set or cross-validation, and the model performance after each iteration is recorded. Based on the performance of the model and the experimental requirements, decide whether to continue the next active learning and self-training iteration. When the third text classification model of this round meets the training stop condition, the training is terminated, and the third text classification model of this round is used as the final model for text classification. Among them, the training stop condition can be set according to actual needs, for example, the number of the fourth text exceeds the third quantity threshold, or the performance of the third classification model reaches the expected level, or the number of training rounds reaches the round number threshold, etc., which is not limited in the embodiments of the present application.

[0099] The training method of the text classification model provided in the embodiment of the present application first uses the labeled first text in the training data to adjust the parameters of the first classification model, and the performance of the obtained second text classification model is improved compared with the first text classification model; on this basis, a self-training method is introduced, that is, the second text classification model classifies the unlabeled second text in the training data to generate corresponding pseudo-labels for the unlabeled second text, and selects the fourth text and its pseudo-label with higher pseudo-label accuracy, and combines the first text and its label in the training data to adjust the parameters of the second text classification model to obtain a third text classification model.

[0100] In this way, with the help of limited annotated text and computing resources, the number of texts and their labels in the training data used for model training can be gradually increased, the training data can be enhanced, thereby improving the generalization ability of the model, and obtaining a third text classification model with high accuracy and high generalization ability, and the amount of annotated data required in the model training process can be minimized.

[0101] In addition, the selection of the fourth text and its pseudo-label is based on the pseudo-label and confidence of the second text and the label of the third text similar to the second text in the training data. This not only reduces the amount of calculation and is efficient, but also ensures that the selected fourth text has high confidence and consistency, reduces the introduction of label noise, and further improves the accuracy and generalization ability of the third text classification model.

[0102] Please refer to Figure 2 , is a flow chart of a method for training a text classification model provided in another embodiment of the present application, the method comprising the following steps: S202: Classify the unlabeled fifth text in the training data based on the first text classification model to obtain a predicted category and confidence level of the fifth text.

[0103] The fifth text is input into the first text classification model for classification, and the predicted category and confidence of the fifth text can be obtained.

[0104] S204: If the confidence of the fifth text is less than the second confidence threshold, label the fifth text based on the predicted category of the fifth text.

[0105] For example, the predicted category of the fifth text may be displayed to annotators, who may refer to the fifth predicted category and annotate the fifth text with a label indicating the category using an annotation tool (such as Oracle).

[0106] S206: Use the fifth text after the label as the first text.

[0107] S208: Based on the annotated first text in the training data and the label of the first text, adjust the parameters of the first text classification model to obtain a second text classification model.

[0108] The specific implementation of the above S208 is Figure 1 The specific implementation of S102 in the illustrated embodiment is similar and will not be described again.

[0109] S210, classifying unlabeled second text in the training data based on the second text classification model to obtain a label and confidence of the second text.

[0110] The specific implementation of S210 is the same as Figure 1 The specific implementation of S104 in the illustrated embodiment is similar and will not be described again.

[0111] S212: Determine a fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data.

[0112] The specific implementation of S212 is the same as Figure 1The specific implementation of S106 in the illustrated embodiment is similar and will not be described again.

[0113] S214: Based on the label of the first text and the label of the fourth text, adjust the parameters of the second text classification model to obtain a third text classification model.

[0114] The specific implementation of the above S214 is Figure 1 The specific implementation of S108 in the illustrated embodiment is similar and will not be described again.

[0115] The training method of the text classification model provided in the embodiment of the present application combines self-training with active learning. Before adjusting the parameters of the first text classification model, the first text classification model first classifies the fifth text in the training data by active learning, selects the fifth text with a lower confidence level for manual annotation, and uses the annotated fifth text as the first text. On the one hand, compared with annotating all the unannotated texts in the training data, the amount of annotation can be reduced; on the other hand, the fifth text with a lower confidence level can usually provide more information for the first text classification model, which helps to improve the model training effect.

[0116] Please refer to Figure 3 , is a flowchart of a method for training a text classification model provided in another embodiment of the present application, the method comprising the following steps: S302, dividing the training data into a labeled data set and an unlabeled data set.

[0117] Initially, the labeled data set contains a small amount of manually labeled first texts, and the unlabeled data set contains a large amount of unlabeled second texts.

[0118] S304: Use the first text in the annotated data set and the label of the first text to adjust the parameters of the language model to obtain a first text classification model.

[0119] Among them, the language model can adopt any model with text processing capabilities, such as SetFit, BERT, etc., which is not limited in the embodiments of the present application.

[0120] S306: Classify the second text in the unlabeled data set based on the first text classification model to obtain a predicted category and confidence level of the second text.

[0121] S308: Select the second text whose confidence is less than the second confidence threshold, and mark it with a corresponding label.

[0122] S310: Add the annotated second text as a new first text to the annotated data set, and delete the annotated second text from the unannotated data set.

[0123] S312: Based on the first text in the new annotated data set and the label of the first text, adjust the parameters of the first text classification model to obtain a second text classification model.

[0124] S314: Classify the second text in the unlabeled data set based on the second text classification model to obtain a label and confidence level of the second text.

[0125] S316 , determining a fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data.

[0126] S318: Add the fourth text as a new first text to the annotated data set, and delete the fourth text from the unannotated data set.

[0127] S320: Based on the label of the first text and the label of the fourth text, adjust the parameters of the second text classification model to obtain a third text classification model.

[0128] If the training stop condition is not met, the above S306 to S320 are repeatedly executed. If the training stop condition is met, the training is stopped, and the third text classification model obtained by the last round of training is used as the final model for text classification.

[0129] The training method for the text classification model provided in the embodiment of the present application first uses a small amount of annotated first text and its labels to adjust the parameters of the language model before the first classification model performs active learning to obtain the first text classification model. This can accelerate the model convergence speed during the training process and improve the model training efficiency.

[0130] Based on the third text classification model trained by the above text classification model training method, the present application embodiment also provides a text classification method. Figure 4 , is a flowchart of a text classification method provided by an embodiment of the present application, the method comprising the following steps: S402, obtaining the target text.

[0131] S404: classify the target text based on the third text classification model to obtain a predicted category of the target text.

[0132] The text classification method provided in the embodiment of the present application uses the third text classification model obtained by the above-mentioned training method to classify the target text, and the obtained prediction category has high reliability and accuracy.

[0133] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0134] Based on the same inventive concept, the present application embodiment also provides a training device for a text classification model. Figure 5 , is a schematic diagram of a text classification model training device provided by an embodiment of the present application. In a software implementation, the text classification model training device 500 includes: a first adjustment module 510, a first classification module 520, a determination module 530 and a second adjustment module 540.

[0135] The first adjustment module 510 is used to adjust the parameters of the first text classification model based on the annotated first text in the training data and the label of the first text to obtain a second text classification model.

[0136] The first classification module 520 is used to classify the unlabeled second text in the training data based on the second text classification model to obtain the label and confidence of the second text.

[0137] The determination module 530 is configured to determine a fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data.

[0138] The second adjustment module 540 is used to adjust the parameters of the second text classification model based on the label of the first text and the label of the fourth text to obtain a third text classification model.

[0139] In another embodiment, In another embodiment, the determining module is used to: If the number of the third texts corresponding to the same label as the second text is greater than a first number threshold, and the confidence of the second text is greater than a first confidence threshold, the second text is determined to be a fourth text.

[0140] In another embodiment, the third text is determined by: Determining a distance between the second text and the text with the label in the training data; Based on the distance, clustering the second text and the text with labels in the training data to obtain a plurality of clusters, each cluster containing at least one text; A plurality of texts are selected from the cluster where the second text is located as third texts similar to the second text.

[0141] In another embodiment, the second adjustment module is used to: Based on the second text classification model, classify the first text to obtain a predicted category of the first text, and classify the fourth text to obtain a predicted category of the fourth text; Determining a first loss based on the label of the first text and the predicted category of the first text; Determining a second loss based on the label of the fourth text, the predicted category of the fourth text, and the weight of the fourth text; Based on the first loss and the second loss, the parameters of the second text classification model are adjusted to obtain a third text classification model.

[0142] In another embodiment, the label of the fourth text represents one of a plurality of preset categories; The weight of the fourth text is determined in the following manner: Determining the number of second texts in the category based on the label of the second text; Determining the weight of each category based on the number of second texts under each category; The weight of the fourth text is determined based on the weight of the category represented by the label of the fourth text.

[0143] In another embodiment, the second adjustment module performs the following steps when determining the weight of each category based on the number of second texts in each category: determining a first ratio between the amount of the second text and the amount of the plurality of categories; For each category, a weight of the category is determined based on a difference between the first ratio and the number of second texts under the category.

[0144] In another embodiment, the second adjustment module performs the following steps when determining the weight of the category based on the difference between the first ratio and the number of second texts under the category for each category: For each category, determining a maximum value between the second text quantity under the category and a second quantity threshold; determining a second ratio between the first ratio and the maximum value; The second ratio is mapped into a preset value range to obtain the weight of the category.

[0145] In another embodiment, the parameters of the second text classification model include the weight ratio of the fourth text; When determining the weight of the fourth text based on the weight of the category represented by the label of the fourth text, the second adjustment module performs the following steps: The product of the weight of the category represented by the label of the fourth text and the proportion is determined as the weight of the fourth text.

[0146] In another embodiment, the first classification module is further used to classify the unlabeled fifth text in the training data based on the first text classification model to obtain the predicted category and confidence of the fifth text; The training device of the text classification model also includes: The labeling module is used for labeling the fifth text based on the predicted category of the fifth text if the confidence of the fifth text is less than a second confidence threshold, and using the labeled fifth text as the first text.

[0147] Obviously, the text classification model training device 500 provided in the embodiment of the present application can be used as the above Figure 1 The execution body of the training method of the text classification model shown in FIG. Figure 1 The functions realized are as follows. Since the principles are the same, they will not be described in detail.

[0148] Based on the same inventive concept, the present application embodiment also provides a text classification device. Figure 6 , is a schematic diagram of a text classification device provided by an embodiment of the present application. In a software implementation, the text classification device 600 includes: an acquisition module 610 and a second classification module 620 .

[0149] The acquisition module 610 is used to acquire the target text.

[0150] The second classification module 620 is used to classify the target text based on a third text classification model to obtain a predicted category of the target text; wherein the third text classification model is trained based on the training method of the text classification model provided in the embodiment of the present application.

[0151] Obviously, the text classification device 600 provided in the embodiment of the present application can be used as the above Figure 4 The execution body of the voiceprint processing method shown in the figure can thus realize the text classification device in Figure 4 The functions realized are as follows. Since the principles are the same, they will not be described in detail.

[0152] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 7 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0153] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0154] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0155] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a training device for the text classification model at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations: Based on the annotated first text in the training data and the label of the first text, adjusting the parameters of the first text classification model to obtain a second text classification model; Classifying the unlabeled second text in the training data based on the second text classification model to obtain a label and a confidence level of the second text; Determine a fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data; Based on the label of the first text and the label of the fourth text, the parameters of the second text classification model are adjusted to obtain a third text classification model.

[0156] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a text classification device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations: Get the target text; The target text is classified based on a third text classification model to obtain a predicted category of the target text; wherein the third text classification model is trained based on the training method of the text classification model provided in an embodiment of the present application.

[0157] The above application Figure 1 The method performed by the training device of the text classification model disclosed in the embodiment shown, or the above-mentioned method of the present application Figure 4 The method performed by the text classification device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0158] The electronic device may also perform Figure 1 The method is used to implement the training device of the text classification model. Figure 1 , Figure 2 , Figure 3The functions of the embodiments shown, or the electronic device may also perform Figure 4 The method is implemented in Figure 4 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0159] Of course, in addition to software implementation, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0160] The embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a portable electronic device including a plurality of application programs, enable the portable electronic device to execute Figure 1 The method of the embodiment shown is specifically used to perform the following operations: Based on the annotated first text in the training data and the label of the first text, adjusting the parameters of the first text classification model to obtain a second text classification model; Classifying the unlabeled second text in the training data based on the second text classification model to obtain a label and a confidence level of the second text; Determine a fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data; Based on the label of the first text and the label of the fourth text, the parameters of the second text classification model are adjusted to obtain a third text classification model.

[0161] Alternatively, when the instruction is executed by a portable electronic device including a plurality of application programs, the portable electronic device can execute Figure 4 The method of the embodiment shown is specifically used to perform the following operations: Get the target text; The target text is classified based on a third text classification model to obtain a predicted category of the target text; wherein the third text classification model is trained based on the training method of the text classification model provided in an embodiment of the present application.

[0162] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0163] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0164] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0165] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0166] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A training method for a text classification model, characterized in that: include: Based on the annotated first text in the training data and the label of the first text, adjusting the parameters of the first text classification model to obtain a second text classification model; Classifying the unlabeled second text in the training data based on the second text classification model to obtain a label and a confidence level of the second text; Determine a fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data; Based on the label of the first text and the label of the fourth text, the parameters of the second text classification model are adjusted to obtain a third text classification model.

2. The method according to claim 1, characterized in that: The determining of the fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data comprises: If the number of the third texts corresponding to the same label as the second text is greater than a first number threshold, and the confidence of the second text is greater than a first confidence threshold, the second text is determined to be a fourth text.

3. The method according to claim 1, characterized in that The third text is determined in the following manner: Determining a distance between the second text and the text with the label in the training data; Based on the distance, clustering the second text and the text with labels in the training data to obtain a plurality of clusters, each cluster containing at least one text; A plurality of texts are selected from the cluster where the second text is located as third texts similar to the second text.

4. The method according to claim 1, characterized in that: The step of adjusting the parameters of the second text classification model based on the label of the first text and the label of the fourth text to obtain a third text classification model includes: Based on the second text classification model, classify the first text to obtain a predicted category of the first text, and classify the fourth text to obtain a predicted category of the fourth text; Determining a first loss based on the label of the first text and the predicted category of the first text; Determining a second loss based on the label of the fourth text, the predicted category of the fourth text, and the weight of the fourth text; Based on the first loss and the second loss, the parameters of the second text classification model are adjusted to obtain a third text classification model.

5. The method according to claim 4, characterized in that The label of the fourth text represents one of the preset multiple categories; The weight of the fourth text is determined in the following manner: Determining the number of second texts in the category based on the label of the second text; Determining the weight of each category based on the number of second texts under each category; The weight of the fourth text is determined based on the weight of the category represented by the label of the fourth text.

6. The method according to claim 5, characterized in that The step of determining the weight of each category based on the number of second texts in each category includes: determining a first ratio between the amount of the second text and the amount of the plurality of categories; For each category, a weight of the category is determined based on a difference between the first ratio and the number of second texts under the category.

7. The method according to claim 6, characterized in that The step of determining the weight of each category based on a difference between the first ratio and the number of second texts under the category comprises: For each category, determining a maximum value between the second text quantity under the category and a second quantity threshold; determining a second ratio between the first ratio and the maximum value; The second ratio is mapped into a preset value range to obtain the weight of the category.

8. The method according to claim 6, characterized in that The parameters of the second text classification model include the weight ratio of the fourth text; The step of determining the weight of the fourth text based on the weight of the category represented by the label of the fourth text comprises: The product of the weight of the category represented by the label of the fourth text and the proportion is determined as the weight of the fourth text.

9. The method according to any one of claims 1 to 8, characterized in that Before adjusting the parameters of the first text classification model based on the first text annotated in the training data and the label of the first text to obtain the second text classification model, the method further includes: Classifying the unlabeled fifth text in the training data based on the first text classification model to obtain a predicted category and confidence level of the fifth text; If the confidence of the fifth text is less than a second confidence threshold, a label is added to the fifth text based on the predicted category of the fifth text, and the labeled fifth text is used as the first text.

10. A text classification method, characterized in that: include: Get the target text; The target text is classified based on a third text classification model to obtain a predicted category of the target text; wherein the third text classification model is trained based on the training method of the text classification model described in any one of claims 1 to 9.

11. A training device for a text classification model, characterized in that: include: A first adjustment module, configured to adjust parameters of the first text classification model based on the annotated first text in the training data and the label of the first text, so as to obtain a second text classification model; A first classification module, used for classifying the unlabeled second text in the training data based on the second text classification model to obtain a label and a confidence level of the second text; A determination module, configured to determine a fourth text from the second text based on the label and confidence of the second text and the label of a third text similar to the second text in the training data; The second adjustment module is used to adjust the parameters of the second text classification model based on the label of the first text and the label of the fourth text to obtain a third text classification model.

12. A text classification device, characterized in that: include: An acquisition module is used to acquire the target text; The second classification module is used to classify the target text based on a third text classification model to obtain a predicted category of the target text; wherein the third text classification model is trained based on the training method of the text classification model described in any one of claims 1 to 9.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the training method of the text classification model as described in any one of claims 1 to 9; or, the processor is configured to execute the instructions to implement the text classification method as described in claim 10.

14. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method of a text classification model as described in any one of claims 1 to 9; or, when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the text classification method as described in claim 10.

15. A computer program product, characterized in that The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps in the training method for a text classification model as described in any one of claims 1 to 9 or the text classification method as described in claim 10.

Images