Text classification model training method and device and text classification method and device

Abstract

The embodiment of the invention provides a text classification model training method and device and a text classification method and device.In the text classification model training method, a training set is obtained, any first training text is input into a text classification model, and one main probability and m additional probabilities are obtained through a main classification network and m additional classification networks in the text classification model respectively. And according to the m additional probabilities of each training text, each definite text is selected from the training set to form a definite set. And based on the predicted loss of each definite text, selecting target texts with a corresponding number of threshold values from the definite set category by category to form a first clean set, and based on other definite texts except the target texts in the definite set, forming a first noise set. And obtaining a prediction label of each first noise text based on the first clean set by using the large model. And training a text classification model according to each first noise text and the prediction label thereof, and each first clean text and the category label thereof.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of machine learning, and in particular, to a text classification model training, text classification method and device. Background Art

[0002] Learning with noisy labeled samples has become an attractive topic because it is usually easier to obtain inaccurately labeled samples. At present, many methods for learning with noisy labeled samples have been proposed, such as actively learning a small language model to re-determine the labels of samples, but this scheme still requires human participation. Another example is to use the contextual learning ability of a large language model (LLM) to re-determine the labels of samples, but high-quality contextual learning examples need to be generated based on clean samples, and the existing screening schemes for clean samples are usually inaccurate, which will affect the effectiveness of the trained model. Summary of the invention

[0003] One or more embodiments of this specification describe a method for training a text classification model, which can be used to train an effective text classification model based on a training set containing noisy samples.

[0004] In a first aspect, a training method for a text classification model is provided, wherein the text classification model includes a main classification network and m additional classification networks; the method includes:

[0005] Obtain a training set, in which each training text has a category label;

[0006] Input any first training text into the text classification model, and obtain the main probability and m additional probabilities that the first training text belongs to the corresponding category label through the main classification network and the m additional classification networks respectively;

[0007] According to the m additional probabilities of each training text, each confident text is selected from the training set to form a confident set;

[0008] Based on the prediction loss of each confident text, target texts of a corresponding number threshold are selected from the confident set by category to form a first clean set; based on other confident texts in the confident set except the target text, a first noise set is formed; the prediction loss of any confident text is determined according to the main probability and category label of the confident text;

[0009] Using the large model, based on the first clean set, obtaining a predicted label of each first noise text in the first noise set;

[0010] The text classification model is trained according to the first noise texts and their predicted labels, and the first clean texts in the first clean set and their category labels.

[0011] In a second aspect, a text classification method is provided, comprising:

[0012] Get the target text to be classified;

[0013] Obtain a text classification model, where the text classification model is trained according to the method of the first aspect above;

[0014] Inputting the target text into the text classification model, and obtaining k probabilities corresponding to k categories through the main classification network therein;

[0015] The category corresponding to the maximum probability among the k probabilities is taken as the target category to which the target text belongs.

[0016] In a third aspect, a training device for a text classification model is provided, wherein the text classification model includes a main classification network and m additional classification networks; the device includes:

[0017] An acquisition unit, used for acquiring a training set, wherein each training text has a category label;

[0018] An input unit, used to input any first training text into the text classification model, and obtain a main probability and m additional probabilities that the first training text belongs to a corresponding category label through the main classification network and the m additional classification networks respectively;

[0019] A selection unit, configured to select each confident text from the training set to form a confident set according to the m additional probabilities of each training text;

[0020] A partitioning unit is used to select target texts of a corresponding number threshold from the confidence set by category based on the prediction loss of each confidence text to form a first clean set; a first noise set is formed based on other confidence texts in the confidence set except the target text; the prediction loss of any confidence text is determined according to the main probability and category label of the confidence text;

[0021] A prediction unit, configured to obtain a prediction label of each first noise text in the first noise set based on the first clean set by using a large model;

[0022] A training unit is used to train the text classification model according to the first noise texts and their predicted labels, and the first clean texts in the first clean set and their category labels.

[0023] In a fourth aspect, a text classification device is provided, comprising:

[0024] An acquisition unit, used for acquiring a target text to be classified;

[0025] The acquisition unit is further used to acquire a text classification model, where the text classification model is trained according to the method of the first aspect above;

[0026] An input unit, used for inputting the target text into the text classification model, and obtaining k probabilities corresponding to k categories through the main classification network therein;

[0027] A determination unit is used to take the category corresponding to the maximum probability among the k probabilities as the target category to which the target text belongs.

[0028] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method of the first or second aspect.

[0029] In a sixth aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of the first or second aspect is implemented.

[0030] The training method of the text classification model provided by one or more embodiments of the present specification is, first, based on multiple additional probabilities output by multiple additional classification networks for training texts, each confident text is selected from the training set to form a confident set, thereby improving the reliability of the confident set. In addition, after obtaining the confident set, the present scheme selects target texts of a corresponding number threshold from the confident set by category to form a clean set, which helps to maintain the balance of samples of each category in the clean set, thereby improving the accuracy of the clean set. It should be understood that when the accuracy of the clean set is improved, the accuracy of the large model's prediction of the noise sample label can be guaranteed, thereby improving the accuracy of the trained text classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 A schematic diagram of an implementation scenario of an embodiment disclosed in this specification;

[0033] Figure 2 A flowchart of a method for training a text classification model according to an embodiment of the present specification is shown;

[0034] Figure 3 A schematic diagram of a text classification model in an example of this specification is shown;

[0035] Figure 4 A schematic diagram showing a method of selecting target text by category in an example of this specification;

[0036] Figure 5 A schematic diagram of a training method for a text classification model according to an embodiment of the present specification is shown;

[0037] Figure 6 A schematic diagram of a training device for a text classification model according to an embodiment of the present specification is shown;

[0038] Figure 7 A schematic diagram of a text classification device according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0039] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0040] In the current clean sample screening scheme, it is usually assumed that clean samples have a smaller mean prediction loss, so a global fixed threshold is set to screen clean samples, but this sample screening scheme is inaccurate. Specifically, the inventor of this scheme observed that the corresponding prediction losses of clean samples (samples without noise labels) and noise samples (samples with noise labels) change dynamically during the model training process. In addition, it is also found that the loss distribution of each category is different. Therefore, currently setting a global fixed threshold to screen clean samples cannot handle the problem of learning samples with noisy labels under unbalanced training sets well.

[0041] To this end, this solution proposes that, first, based on the multiple additional probabilities output by multiple additional classification networks for the training texts, each confident text is selected from the training set to form a confident set. Then, the target texts corresponding to the number threshold are selected from each category of the confident set to form a clean set, which helps to maintain the balance of samples of each category in the clean set, thereby improving the accuracy of the clean set.

[0042] Figure 1 A schematic diagram of an implementation scenario of an embodiment disclosed in this specification. Figure 1 In the text classification model, the text classification model includes 1 main classification network and m additional classification networks. Each classification network is used to predict the probability that the training text belongs to the corresponding category label. Among them, the probability predicted by the main classification network is called the main probability, which is used to calculate the prediction loss; the probability predicted by the additional classification network is called the additional probability, which is used to divide the training set.

[0043] The training process of the above text classification model can be as follows: input each training text in the training set into the text classification model to obtain 1 main probability and m additional probabilities for each training text. Then, according to the m additional probabilities of each training text, each confident text can be selected from the training set to form a confident set, and based on the prediction loss of each confident text, the confident set can be divided into a clean set and a noise set. After that, using the large model, based on the clean set, the predicted labels of each noise text in the noise set are obtained, and the text classification model is trained according to each noise text and its predicted label, and each clean text in the clean set and its category label.

[0044] Figure 2 The flowchart of the training method of the text classification model according to one embodiment of the present specification is shown. The method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. It should be noted that the method includes multiple rounds of iterations. Figure 2 The method steps included in the tth (t is a positive integer) iteration are shown. It can be understood that by repeatedly executing the steps shown therein, multiple rounds of parameter adjustment of the text classification model can be achieved, and then the text classification model after the last round of parameter adjustment is used as the final use model. Figure 2 As shown, the method may include the following steps:

[0045] Step S202, obtaining a training set, wherein each training text has a category label.

[0046] For the sake of simplicity and clarity, the i-th training text in the above training set can be recorded as (x i ,y i ), where x i Represents the text features of the i-th training text, including but not limited to text content, text metadata (such as author, release time, language, and length, etc.). In addition, it can also include words extracted from the sentiment dictionary, etc. i Represents the category label of the i-th training text, which is selected from the predetermined k categories. Typically, in the scenario of sentiment polarity classification, the k categories include positive and negative. At this time, two different label values, such as 0 and 1, can be used to represent the positive and negative categories respectively. In some scenarios, the classification of text can also be multi-classification. For example, in the SMS text classification scenario, the SMS text can be classified into multiple industries. At this time, the multiple industries can correspond to k categories, and k can be any set integer.

[0047] Step S204, input any training text into the text classification model, and obtain the main probability and m additional probabilities that the training text belongs to the corresponding category label through the main classification network and m additional classification networks therein.

[0048] The main classification network and the m additional classification networks may be classification networks for performing classification of k categories, where m is a hyperparameter and k is a positive integer.

[0049] It should be understood that when the above-mentioned main classification network is used to perform k-category classification, after any training text is input into the text classification model, k probabilities corresponding to k categories can be obtained through the main classification network, and the above-mentioned main probability is the probability of the category label corresponding to the training text among the k probabilities.

[0050] Similarly, when the additional classification network is used to perform k category classification, after any training text is input into the text classification model, k probabilities corresponding to k categories can be obtained through the additional classification network, and the additional probability is the probability of the category label corresponding to the training text among the k probabilities.

[0051] Figure 3 A schematic diagram of a text classification model in one example of this specification is shown. Figure 3 In the text classification model, the text representation network, the main classification network and m additional classification networks are included. The text representation network can be implemented as a Bert model, a Transformer encoder, etc. The main classification network or a single additional classification network can include one or more fully connected layers and an output layer. More specifically, the output layer uses a softmax function to operate on the input vector.

[0052] Specifically, for any training text, after inputting it into the text classification model, the text representation network can be used to embed the training text to obtain the corresponding text vector. After that, the main classification network can be used to determine the above main probability based on the text vector. For example, the text vector is first processed using each fully connected layer, and then the processed text vector is operated using the softmax function to obtain k probabilities corresponding to k categories, and the probability of the category label corresponding to the training text among the k probabilities is determined as the main probability. Similarly, m additional classification networks can be used to determine m additional probabilities based on the above text vector.

[0053] Similarly, the main probability and m additional probabilities corresponding to each training text can be obtained.

[0054] Take the i-th training text (x i ,y i ) as an example, the corresponding main probability can be expressed as: p G (y i ;x i ), and the corresponding m additional probabilities can be expressed as: It should be understood that the m additional probabilities correspond to m additional classification networks: g1,…,g m .

[0055] In this solution, in each of the above-mentioned additional classification networks, a corresponding dynamic threshold can be set for each training text. The reason why it is called a dynamic threshold here is that the threshold will be updated after each round of iteration.

[0056] For example, for the i-th training text (x i ,y i ), in the m additional classification networks, the m dynamic thresholds set for them can be expressed as: Where t represents the tth iteration.

[0057] The following is an explanation of why this solution sets a dynamic threshold for each training text:

[0058] For categories with a small number of samples (also called tail categories), the inventors of this solution observed that the prediction loss of noise samples will continue to decrease during the training process. This observed phenomenon may be attributed to the memory effect of the model, that is, the model's memory strength for samples with a given label becomes stronger as learning increases. In other words, as training proceeds, the noise samples will gradually be memorized, and their prediction loss will decrease accordingly. In this case, setting a fixed threshold is not the best choice. Therefore, this solution selects a confidence set based on a dynamic threshold.

[0059] It should be understood that Figure 3 This is just an exemplary description. In practice, the text classification model may also include only one main classification network and m additional classification networks. In this case, the text vector of the training text may be determined first, and then the text vector may be input into the text classification model.

[0060] Among them, the process of determining the text vector of the training text is as follows: first, each text feature of the training text is embedded to obtain the corresponding embedding vector. At present, there are many word embedding algorithms that implement word vector training based on a large amount of text corpus. Therefore, the embedding vector of each text feature of the training text can be directly determined by consulting the trained word vector table. After that, the embedding vector can be further processed using a neural network to obtain the text vector of the training text. In a specific embodiment, the neural network here can be a convolutional neural network CNN, or a recurrent neural network RNN.

[0061] It should be noted that the traditional text classification model only includes a single classification network. When selecting samples based on the single classification network, the selected samples are usually biased samples because they are selected only based on the confidence provided by itself, which will affect the accuracy of the selected confident text. To this end, this solution constructs a text classification model that includes multiple classification networks at the same time, thereby achieving independent parameter learning and sample selection in a single classification network. It should be understood that by combining the different evidence (confidence) provided by the multiple additional classification networks to memorize samples, the accuracy of the selected confident text can be greatly improved.

[0062] Step S206, selecting each confident text from the training set to form a confident set according to the m additional probabilities of each training text.

[0063] Specifically, for any training text, when the m additional probabilities of the training text are respectively greater than the m current thresholds corresponding to the training text in the corresponding m additional classification networks, the training text is taken as one of the confident texts.

[0064] In one example, the belief set E can be expressed as follows:

[0065]

[0066] And, the non-confident set U can be expressed as follows:

[0067]

[0068] Of course, in practice, among the m additional probabilities of the training text, if the additional probabilities exceeding a predetermined number or a predetermined proportion are greater than the corresponding current threshold, it can be selected as the confident text, and this specification does not limit this.

[0069] It should be understood that the principle of selecting the confident text in this scheme is: if multiple additional classification networks simultaneously memorize the training text with high confidence, the training text is considered to be confident. Otherwise, if only one or a few additional classification networks output an additional probability exceeding the threshold, the training text is considered to be unconfident.

[0070] It should be noted that as training progresses, the model will eventually memorize the noise samples, which means that multiple additional classification networks may memorize the noise samples at the same time during training, so it is necessary to further distinguish whether the samples in the confidence set E are clean or noisy.

[0071] Step S208, based on the prediction loss of each confident text, select target texts of a corresponding number threshold from the confident set by category to form a clean set C1, and form a noise set N1 based on other confident texts in the confident set except the target texts selected for each category.

[0072] It should be noted that the previous small loss selection method shows that the prediction loss of clean samples has a small mean, and a global fixed threshold is set to select clean samples. However, the inventor of this solution observed that the loss distribution of categories with a large number of samples (also called head categories) and categories with a small number of samples (also called tail categories) is different. Therefore, the prediction losses of samples of different categories may not be comparable, so a more appropriate way is to select samples one by one according to the same category, rather than uniformly setting a global fixed threshold.

[0073] Specifically, for any certain text, the corresponding prediction loss can be calculated based on the main probability and category label of the certain text using the cross entropy loss function. In this way, the prediction loss of each certain text can be calculated. Then, the prediction loss of each certain text can be normalized to bring them into a common scale, so that the comparison of prediction losses between certain texts is meaningful. In one example, the prediction loss of each certain text can be mapped to the interval [0,1].

[0074] Afterwards, the confidence set E can be divided into k text groups according to the category labels, where k is the number of category labels to which the texts contained in the confidence set E belong. The category labels of the confidence texts in the same text group in the k text groups are the same, that is, one text group corresponds to one category label. In an example, the kth text group can be represented as: E k ={(x i ,y i )∈E|y i =k}. Then, for each text group or each category label, a corresponding number threshold can be calculated, and the corresponding target text can be selected according to the number threshold.

[0075] Specifically, for the kth group, the corresponding number threshold φ(k) can be the minimum value of the target value determined based on the number of texts contained in the confidence set and the number of category labels to which the contained texts belong, and the number of texts contained in the kth text group.

[0076] In a more specific embodiment, Among them, N is the number of texts contained in the confidence set, k is the number of category labels to which the texts contained in the confidence set belong, and R is the predefined filtering rate. is the target value, N k is the number of texts contained in the kth text group, and min() is the minimum value function.

[0077] It should be understood that as the training process progresses, N, k, and N k It changes dynamically, so the above number threshold φ(k) is a dynamic threshold.

[0078] Of course, in practice, the number threshold can also be set in other ways, for example, directly setting the number threshold φ(k) to N k , this manual does not limit this.

[0079] Figure 4 A schematic diagram showing the selection of target text by category in an example of this specification. Figure 4 In the example, for the k-th text group, the confident texts can be sorted from small to large according to the prediction loss of each confident text, and the top φ(k) confident texts are selected as the corresponding target texts, that is, the target texts under the k-th category label, which is specifically expressed as: E φ(k) After selecting the corresponding target text for each text group or each category label, each target text corresponding to each category label forms a clean set, which can be expressed as:

[0080] Of course, in practice, each confident text may be sorted from large to small according to the prediction loss, and the φ(k) confident texts with the lowest ranking may be selected as the target texts, which is not limited in this specification.

[0081] As can be seen from the above, compared with the previous small loss selection, this scheme sets the corresponding number threshold φ(k) for each category, and selects φ(k) samples with the minimum prediction loss from each category k to form a set D φ(k) Finally, all sets are merged into In D φ The samples selected from the dataset have a high probability of being clean, which can prevent the samples of the tail category from being ignored during the selection process and ensure that they fully participate in the training process. Therefore, the selection method of this scheme can alleviate the selection bias to a certain extent, enabling the text classification model to fully learn from the samples of the tail category and generate correct predictions.

[0082] In addition, based on the other confident texts in the confident set except for the target texts corresponding to the category labels, a noise set N1 can be formed. In other words, in this scheme, the confident set E can be divided into a clean set C1 and a noise set N1. Similar to the division method of the confident set E, this scheme can divide the non-confident set U into a clean set C2 (also called a difficult set) and a noise set N2.

[0083] Step S210, using the large model, based on the clean set C1, obtain the predicted label of each noise text in the noise set N1.

[0084] In this step, the large model is used to generate new labels for the noisy text. However, the core challenge lies in the construction of contextual learning examples. Since the proportion of noisy text in the clean set C1 is significantly reduced, samples can be selected from the clean set C1 to construct contextual learning examples. At the same time, the proportion of noisy samples in the noisy set N1 is significantly increased, which needs to be corrected by querying the large model.

[0085] Specifically, for any noise text in the noise set N1, a target clean text matching it is selected from the clean set C1 by category. Based on each target clean text under each category label and each category label, the context of the noise text is determined. The noise text and its context are input into the large model, so that it generates a predicted label for the noise text based on the above context.

[0086] It should be understood that the large model here generates the predicted labels of noise samples by learning the association between each target clean text and its category label.

[0087] More specifically, for the noisy text x i , we can calculate x i And each clean text x under the k-th category label in the clean set C1 j (x j ∈C 1,k ,C 1,k ={(x i ,y i )∈C1|y i =k}), and then select p clean texts with the largest similarity (that is, after sorting the clean texts from large to small according to the similarity, the top p clean texts) as the target clean text under the kth category label.

[0088] The similarity mentioned above may include but is not limited to cosine similarity, Euclidean distance, Manhattan distance, Pearson correlation coefficient, etc.

[0089] In practice, we can first determine the noise text x i The text vector of each clean text x j The text vector is then used to calculate the similarity.

[0090] In one embodiment, the noise text x i and each clean text x j The text vector of can be determined by using a neural network. The specific determination process is described above and will not be repeated here.

[0091] In another embodiment, the noise text x i and each clean text x jThe text vector can be determined using a representation model such as a Bert model or a Transformer encoder.

[0092] Similar to the method of selecting the target clean text under the kth category label, the target clean text under each category label can be selected, and then each category label and the target clean text under each category label can be spliced ​​to obtain the noisy text x i context.

[0093] It should be noted that Retrieval Augmented Generation (RAG) has become increasingly popular to enhance the predictive power of large models by retrieving relevant information. Inspired by this, this solution introduces corresponding explanation text for each category label (which can be obtained by querying the search engine). The explanation text can provide relevant class knowledge for the category label, thereby enhancing the predictive power of large models, especially for classification tasks that require a lot of domain expertise.

[0094] Specifically, for any category label l i , for this category label l i The target clean text, the category label l i And the category label l i The explanation texts are spliced ​​to obtain the corresponding splicing results. After that, the splicing results corresponding to each category label are combined to obtain the noise text x i context.

[0095] It should be understood that in practice, the input of the large model may also include a labeling rule. The labeling rule may be generated manually or by using the large model to learn the association between each clean text in the clean set C1 and its category label.

[0096] In summary, the large model can output the predicted label of each noise text based on the context constructed for each noise text in the noise set N1.

[0097] Similarly, the context of each noise text in the noise set N2 can be constructed based on the clean set C1, and the large model can output the predicted label of each noise text in the noise set N2 based on the context.

[0098] Step S212, training a text classification model based on the noise texts and their predicted labels in the N1 noise set, and the clean texts and their category labels in the clean set C1.

[0099] Specifically, the loss L1 can be calculated based on the probability that each clean text in the clean set C1 belongs to the corresponding category label and its category label using the first loss function. And the loss L2 can be calculated based on the probability that each noise text in the noise set N1 belongs to the corresponding predicted label and its predicted label using the first loss function. Afterwards, based on the comprehensive loss of L1 and L2 (for example, summing L1 and L2), the back propagation method is used to calculate the update gradient corresponding to the parameters of the main classification network and the m additional classification networks, and then the parameters of the main classification network and the m additional classification networks are updated based on the update gradient to obtain the trained text classification model.

[0100] Of course, when the text classification model also includes a text representation network, the update gradient corresponding to the parameters of the text representation network can also be calculated, and the parameters of the text representation network can be updated based on it.

[0101] In an example, the first loss function may be a cross entropy loss function, and the second loss function may be a noise-robust loss function such as a symmetric cross entropy loss function or a KL divergence loss function.

[0102] In another embodiment, the noise texts in the noise set N2 and their predicted labels may be combined to train a text classification model.

[0103] Specifically, the second loss function can also be used to calculate the loss L3 based on the probability that each noise text in the noise set N2 belongs to the corresponding predicted label and its predicted label. Afterwards, the sum of L1, L2 and L3 is used as the comprehensive loss, and the back propagation method is used based on the comprehensive loss to calculate the update gradient of the parameters of each classification network, thereby updating the parameters of each classification network.

[0104] It should be noted that since the output of the large model is not necessarily completely correct and hallucinations may exist, the above-mentioned noise-robust loss function is used to calculate the corresponding loss for each noise text in the noise sets N1 and N2.

[0105] In yet another embodiment, the clean texts in the clean set C2 and their category labels may be combined to train a text classification model.

[0106] Specifically, several text pairs can be randomly selected from each clean text in the clean set C2. For any text pair, the two text vectors of the two clean texts are weightedly fused to obtain a fusion vector, and the two category labels of the two clean texts are weightedly fused to obtain a fusion label. The main classification network is used to obtain two probabilities corresponding to the two category labels based on the fusion vector. The loss L4 is determined based on the fusion probability of the two probabilities corresponding to each text pair and its fusion label.

[0107] For example, suppose the randomly selected text pair is: (x i ,y i ) and (x j ,y j ), in addition, it is assumed that max(p(k; x j ))≤max(p(k;x i )), where max(p(k; x j )) is the main classification network for (x j ,y j ) is the maximum probability among the k probabilities output, max(p(k; x i )) is the main classification network for (x i ,y i ) is the maximum probability among the k probabilities output by . Then, the above fusion vector and fusion label are obtained by the following formula:

[0108] e′ i =λ′e i +(1-λ′)e j ;y′ i =λ′y i +(1-λ′)y j ;

[0109] Among them, e i is (x i ,y i )’s text vector, e j is (x j ,y j ), e′ i is the fusion vector; y′ i is a fusion label; λ′=max(λ,1-λ), and λ~Beta(α,α).

[0110] In one example, the loss L4 can be calculated according to the following formula:

[0111]

[0112] Among them, e′ i is the fusion vector, y′ i For fusion tags, are the outputs of the main classification network based on the fusion vector corresponding to y i and j The fused probability of two probabilities, for example, the sum of the two probabilities. is the number of randomly selected text pairs, and N is the number of texts contained in the clean set C2.

[0113] It should be understood that when the above-mentioned loss L4 is also calculated, the sum of L1, L2, L3 and L4 can be used as a comprehensive loss, and based on the comprehensive loss, the back propagation method is used to calculate the update gradient of the parameters of each classification network, thereby updating the parameters of each classification network.

[0114] At this point, the parameter update of the tth round of the text classification model has been completed.

[0115] In addition, as mentioned above, the dynamic threshold set for each training text will be updated after each iteration. Specifically, the m current thresholds of the training text and the m additional probabilities of the training text can be weighted and summed to obtain the m updated thresholds of the training text for the next iteration.

[0116] For example, for the i-th training text (x i ,y i ), the updating method of the m dynamic thresholds corresponding to the m additional classification networks can be as follows:

[0117]

[0118] …

[0119]

[0120] It should be understood that by repeatedly executing the above steps S202 to S212, multiple rounds of iterative updates of the above text classification model can be achieved, and then the text classification model obtained by the last round of iterative updates is used as the final use model. Among them, the end conditions of multiple rounds of iterations may include that the number of iterations reaches a predetermined number, or the model parameters converge, etc.

[0121] In summary, the training method of the text classification model provided in the embodiment of this specification focuses on the learning of the training set containing noise samples and unbalanced sample categories. In order to solve the problems of noise labels and sample category imbalance at the same time, firstly, based on the dynamic threshold set for each training text, each sure text is selected from the training set to form a sure set. After that, based on the dynamic selection by category for the determination set, clean samples and noise samples are selected for use by LLM, and then the clean samples are used to construct context learning examples and the noise samples are re-labeled using LLM to act as an active annotator. In addition, this scheme selects the sure set based on the output of multiple additional classification networks, which can reduce the error accumulation problem in the model training process, thereby making the selected sure set more robust.

[0122] Figure 5 A schematic diagram of a method for training a text classification model according to an embodiment of the present specification is shown. Figure 5In the above method, each training text in the training set is input into the text classification model to obtain 1 main probability and m additional probabilities for each training text. Then, according to the m additional probabilities of each training text, the training set can be divided into a confident set E and an unconfident set U. After that, the confident set E can be divided into a clean set C1 and a noise set N1, and the unconfident set can be divided into a clean set C2 and a noise set N2 according to the same division method. Among them, the clean set C1 is used as the final clean set C to construct context learning examples. The noise set N1 and the noise set N2 constitute the final noise set N, which is re-labeled by the large model. The clean set C2 is used as the final difficult set H. Finally, based on the three different data sets of clean set C, noise set N and difficult set H, the corresponding loss function is used to calculate the corresponding three losses, and the parameters of the text classification model are adjusted based on the comprehensive loss of the three losses.

[0123] It should be understood that after the text classification model is trained, the trained text classification model can be used to perform text classification tasks. The text classification process is described in detail below.

[0124] Get the target text to be classified. Enter the target text through Figure 2 The text classification model trained by the method shown obtains k probabilities corresponding to k categories through the main classification network therein, and the category corresponding to the maximum probability among the k probabilities is used as the target category to which the target text belongs.

[0125] Corresponding to the above-mentioned text classification model training method, an embodiment of the present specification also provides a text classification model training device, such as Figure 6 As shown, the device may include:

[0126] The acquisition unit 602 is used to acquire a training set, in which each training text has a category label.

[0127] The input unit 604 is used to input any first training text into the text classification model, and obtain the main probability and m additional probabilities that the first training text belongs to the corresponding category label through the main classification network and the m additional classification networks respectively.

[0128] The selection unit 606 is used to select each confident text from the training set to form a confident set according to the m additional probabilities of each training text.

[0129] The division unit 608 is used to select target texts of a corresponding number threshold from the confidence set by category based on the prediction loss of each confidence text to form a first clean set, and to form a first noise set based on other confidence texts in the confidence set except the target text. The prediction loss of any confidence text is determined according to the main probability and category label of the confidence text.

[0130] The prediction unit 610 is used to obtain a prediction label of each first noise text in the first noise set based on the first clean set by using the large model.

[0131] The training unit 612 is used to train a text classification model according to each first noise text and its predicted label, and each first clean text in the first clean set and its category label.

[0132] In one embodiment, the text classification model further includes a text representation network; the above device further includes:

[0133] A processing unit 614 is used to process the first training text using a text representation network to obtain a first text vector;

[0134] A determination unit 616, configured to determine a primary probability based on the first text vector using a primary classification network;

[0135] The determination unit 616 is further configured to determine m additional probabilities based on the first text vector by using the m additional classification networks respectively.

[0136] In one embodiment, the selection unit 606 is specifically used to:

[0137] For any second training text, when the m additional probabilities of the second training text are respectively greater than the m current thresholds corresponding to the second training text in the corresponding m additional classification networks, the second training text is taken as one of the confident texts.

[0138] In one embodiment, the apparatus further comprises:

[0139] The summing unit 618 is used to perform weighted summation of the m current thresholds and the m additional probabilities of the second training text respectively to obtain m updated thresholds of the second training text for use in the next round of iteration.

[0140] In one embodiment, the division unit 608 is specifically configured to:

[0141] For any first category label, the first confident texts under the first category label are sorted from small to large according to the corresponding prediction loss, and a first threshold number of first confident texts ranked at the top are selected as the target texts under the first category label.

[0142] The first number threshold is the minimum value of the target value determined based on the first number of texts contained in the confidence set, the second number of category labels to which the contained texts belong, and the third number of each first confidence text.

[0143] In one embodiment, the prediction unit 610 includes:

[0144] A selection submodule 6102 is used to select, for any first noise text in the first noise set, a target clean text that matches it from the first clean set by category;

[0145] A determination submodule 6104 is used to determine the context of the first noise text based on each target clean text under each category label and each category label;

[0146] The input submodule 6106 is used to input the first noise text and the above context into the large model so that it generates a predicted label for the first noise text based on the context.

[0147] Wherein, the determination submodule 6104 is specifically used for:

[0148] For any first category label, concatenate the target clean text under the first category label, the first category label, and the explanation text of the first category label to obtain a first concatenation result;

[0149] The concatenation results corresponding to the category labels are combined to obtain the context of the first noise text.

[0150] In one embodiment, the apparatus further comprises:

[0151] A forming unit 620, configured to form a non-confident set based on each training text except each confident text in the training set;

[0152] The division unit 608 is further configured to divide the non-confident set into a second clean set and a second noise set;

[0153] The prediction unit 610 is further used to obtain a prediction label of each second noise text in the second noise set based on the first clean set by using the large model;

[0154] The training unit 612 is specifically used for:

[0155] A text classification model is trained based on the first and second noisy texts and their predicted labels, and the first clean texts and their category labels.

[0156] In one embodiment, the training unit 612 includes:

[0157] A calculation submodule 6122, configured to calculate a first loss based on each first clean text and its category label using a first loss function;

[0158] The calculation submodule 6122 is further used to calculate a second loss based on the first and second noise texts and their predicted labels using a second loss function;

[0159] The adjustment submodule 6124 is used to adjust the parameters of each classification network based on the first loss and the second loss.

[0160] Among them, the first loss function is the cross entropy loss function, and the second loss function is the symmetric cross entropy loss function or the KL divergence loss function.

[0161] In one embodiment, the apparatus further comprises:

[0162] An extraction unit 622, configured to randomly extract a number of text pairs from each second clean text in the second clean set;

[0163] A fusion unit 624 is used for performing weighted fusion on two text vectors of two second clean texts in any first text pair to obtain a fusion vector, and performing weighted fusion on two category labels of the two second clean texts to obtain a fusion label;

[0164] The acquisition unit 602 is further configured to acquire two probabilities corresponding to two category labels respectively based on the fusion vector through the main classification network; determine a third loss based on the fusion probability of the two probabilities corresponding to each text pair and the fusion label thereof;

[0165] The adjustment submodule 6124 is specifically used for:

[0166] Based on the comprehensive loss of the first, second and third losses, the parameters of each classification network are adjusted.

[0167] The functions of the functional units of the device in the above-mentioned embodiment of this specification can be implemented through the steps of the above-mentioned method embodiment. Therefore, the specific working process of the device provided by one embodiment of this specification will not be repeated here.

[0168] A training device for a text classification model provided in one embodiment of the present specification can train an effective text classification model based on a training set containing noisy samples.

[0169] Corresponding to the above-mentioned text classification method, an embodiment of the present specification also provides a text classification device, such as Figure 7 As shown, the device may include:

[0170] The acquisition unit 702 is used to acquire the target text to be classified.

[0171] The acquisition unit 702 is also used to acquire a text classification model. Figure 2 The method steps shown are obtained by training.

[0172] The input unit 704 is used to input the target text into the text classification model, and obtain k probabilities corresponding to k categories through the main classification network therein.

[0173] The determination unit 706 is configured to take the category corresponding to the maximum probability among the k probabilities as the target category to which the target text belongs.

[0174] The functions of the functional units of the device in the above-mentioned embodiment of this specification can be implemented through the steps of the above-mentioned method embodiment. Therefore, the specific working process of the device provided by one embodiment of this specification will not be repeated here.

[0175] A text classification device provided in an embodiment of the present specification can improve the accuracy of text classification.

[0176] According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 2 The method described.

[0177] According to another embodiment of the present invention, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the Figure 2 The method described.

[0178] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the server embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0179] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0180] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of this specification. It should be understood that the above description is only the specific implementation method of this specification and is not intended to limit the protection scope of this specification. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of this specification should be included in the protection scope of this specification.

Claims

1. A training method for a text classification model, the text classification model comprising a main classification network and m additional classification networks; the method comprising: Obtain a training set, in which each training text has a category label; Input any first training text into the text classification model, and obtain the main probability and m additional probabilities that the first training text belongs to the corresponding category label through the main classification network and the m additional classification networks respectively; According to the m additional probabilities of each training text, each confident text is selected from the training set to form a confident set; Based on the prediction loss of each confident text, target texts of a corresponding number threshold are selected from the confident set by category to form a first clean set; based on other confident texts in the confident set except the target text, a first noise set is formed; the prediction loss of any confident text is determined according to the main probability and category label of the confident text; Using the large model, based on the first clean set, obtaining a predicted label of each first noise text in the first noise set; The text classification model is trained according to the first noise texts and their predicted labels, and the first clean texts in the first clean set and their category labels.

2. The method according to claim 1, wherein: The text classification model also includes a text representation network; the main probability and m additional probabilities are obtained by the following steps: Using the text representation network, processing the first training text to obtain a first text vector; Determining the primary probability based on the first text vector using the primary classification network; The m additional probabilities are determined based on the first text vector using the m additional classification networks respectively.

3. The method according to claim 1, wherein: The step of selecting each confident text from the training set includes: For any second training text, when the m additional probabilities of the second training text are respectively greater than the m current thresholds corresponding to the second training text in the corresponding m additional classification networks, the second training text is taken as one of the confirmed texts.

4. The method according to claim 3, further comprising: The m current thresholds are weighted and summed with the m additional probabilities of the second training text respectively to obtain m updated thresholds of the second training text for use in the next round of iteration.

5. The method according to claim 1, wherein: The step of selecting target texts of a corresponding number threshold from the confidence set by category includes: For any first category label, the first confident texts under the first category label are sorted from small to large according to the corresponding prediction loss, and a first threshold number of first confident texts ranked at the top are selected as the target texts under the first category label.

6. The method according to claim 5, wherein: The first number threshold is the minimum value of the target value determined based on the first number of texts contained in the confidence set and the second number of category labels to which the contained texts belong, and the third number of the first confidence texts.

7. The method according to claim 1, wherein: The obtaining of the predicted label of each first noise text in the first noise set includes: For any first noise text in the first noise set, selecting a target clean text matching it from the first clean set by category; Determining the context of the first noise text based on each target clean text and each category label under each category label; The first noise text and the context are input into the large model so that it generates a predicted label of the first noise text based on the context.

8. The method according to claim 7, wherein: The determining the context of the first noise text comprises: For any first category label, concatenate the target clean text under the first category label, the first category label, and the explanation text of the first category label to obtain a first concatenation result; The concatenation results corresponding to the category labels are combined to obtain the context of the first noise text.

9. The method according to claim 1, further comprising: Forming a non-confident set based on each training text in the training set except the each confident text; Dividing the non-confident set into a second clean set and a second noise set; Using the large model, based on the first clean set, obtaining a predicted label of each second noise text in the second noise set; The training of the text classification model comprises: The text classification model is trained based on the first and second noise texts and their predicted labels, and the first clean texts and their category labels.

10. The method according to claim 9, wherein: The text classification model is trained according to the first and second noise texts and their predicted labels, and the first clean texts and their category labels, including: Using a first loss function, based on each first clean text and its category label, a first loss is calculated; Calculating a second loss based on the first and second noise texts and their predicted labels using a second loss function; Based on the first loss and the second loss, the parameters of each classification network are adjusted.

11. The method according to claim 10, wherein: The first loss function is a cross entropy loss function, and the second loss function is a symmetric cross entropy loss function or a KL divergence loss function.

12. The method according to claim 10, further comprising: Randomly extract a number of text pairs from each second clean text in the second clean set; For any first text pair, weighted fusion is performed on two text vectors of two second clean texts therein to obtain a fusion vector, and weighted fusion is performed on two category labels of the two second clean texts to obtain a fusion label; Obtaining two probabilities corresponding to the two category labels respectively based on the fusion vector through the main classification network; Determine a third loss based on the fusion probability of the two probabilities corresponding to each text pair and its fusion label; The adjusting the parameters of each classification network includes: Based on the comprehensive loss of the first, second and third losses, the parameters of each classification network are adjusted.

13. A text classification method, comprising: Get the target text to be classified; Obtaining a text classification model, wherein the text classification model is trained according to the method as claimed in claim 1; Inputting the target text into the text classification model, and obtaining k probabilities corresponding to k categories through the main classification network therein; The category corresponding to the maximum probability among the k probabilities is taken as the target category to which the target text belongs.

14. A training device for a text classification model, the text classification model comprising a main classification network and m additional classification networks; The device comprises: An acquisition unit, used for acquiring a training set, wherein each training text has a category label; An input unit, used to input any first training text into the text classification model, and obtain a main probability and m additional probabilities that the first training text belongs to a corresponding category label through the main classification network and the m additional classification networks respectively; A selection unit, configured to select each confident text from the training set to form a confident set according to the m additional probabilities of each training text; A partitioning unit is used to select target texts of a corresponding number threshold from the confidence set by category based on the prediction loss of each confidence text to form a first clean set; a first noise set is formed based on other confidence texts in the confidence set except the target text; the prediction loss of any confidence text is determined according to the main probability and category label of the confidence text; A prediction unit, configured to obtain a prediction label of each first noise text in the first noise set based on the first clean set by using a large model; A training unit is used to train the text classification model according to the first noise texts and their predicted labels, and the first clean texts in the first clean set and their category labels.

15. The device according to claim 14, wherein: The text classification model also includes a text representation network; the device also includes: A processing unit, configured to process the first training text using the text representation network to obtain a first text vector; A determination unit, configured to determine the main probability based on the first text vector using the main classification network; The determination unit is further configured to respectively use the m additional classification networks to determine the m additional probabilities based on the first text vector.

16. The device according to claim 14, wherein: The selection unit is specifically used for: For any second training text, when the m additional probabilities of the second training text are respectively greater than the m current thresholds corresponding to the second training text in the corresponding m additional classification networks, the second training text is taken as one of the confirmed texts.

17. The apparatus according to claim 16, further comprising: The summing unit is used to perform weighted summation on the m current thresholds and the m additional probabilities of the second training text respectively to obtain m updated thresholds of the second training text for the next round of iteration.

18. The device according to claim 14, wherein: The division unit is specifically used for: For any first category label, the first confident texts under the first category label are sorted from small to large according to the corresponding prediction loss, and a first threshold number of first confident texts ranked at the top are selected as the target texts under the first category label.

19. The device according to claim 14, wherein: The prediction unit comprises: A selection submodule, configured to select, for any first noisy text in the first noisy set, a target clean text that matches the first noisy text from the first clean set by category; A determination submodule, used to determine the context of the first noise text based on each target clean text under each category label and each category label; An input submodule is used to input the first noise text and the context into the large model so that it generates a predicted label for the first noise text based on the context.

20. The apparatus of claim 14, further comprising: A forming unit, used for forming a non-confident set based on each training text in the training set except the each confident text; The division unit is further used to divide the non-confident set into a second clean set and a second noise set; The prediction unit is further used to obtain a prediction label of each second noise text in the second noise set based on the first clean set by using the large model; The training unit is specifically used for: The text classification model is trained based on the first and second noise texts and their predicted labels, and the first clean texts and their category labels.

21. The device according to claim 20, wherein: The training unit comprises: A calculation submodule, configured to calculate a first loss based on each first clean text and its category label by using a first loss function; The calculation submodule is further used to calculate a second loss based on the first and second noise texts and their predicted labels using a second loss function; The adjustment submodule is used to adjust the parameters of each classification network based on the first loss and the second loss.

22. The apparatus according to claim 21, further comprising: An extraction unit, used for randomly extracting a number of text pairs from each second clean text in the second clean set; A fusion unit is used for performing weighted fusion on two text vectors of two second clean texts in any first text pair to obtain a fusion vector, and performing weighted fusion on two category labels of the two second clean texts to obtain a fusion label; The acquisition unit is further configured to acquire two probabilities corresponding to the two category labels respectively based on the fusion vector through the main classification network; and determine a third loss based on the fusion probability of the two probabilities corresponding to each text pair and its fusion label; The adjustment submodule is specifically used for: Based on the comprehensive loss of the first, second and third losses, the parameters of each classification network are adjusted.

23. A text classification device, comprising: An acquisition unit, used for acquiring a target text to be classified; The acquisition unit is further used to acquire a text classification model, where the text classification model is trained according to the method according to claim 1; An input unit, used for inputting the target text into the text classification model, and obtaining k probabilities corresponding to k categories through the main classification network therein; A determination unit is used to take the category corresponding to the maximum probability among the k probabilities as the target category to which the target text belongs.

24. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 13.

25. A computing device comprising a memory and a processor, wherein: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 13 is implemented.

Images