Text classification model training and text classification method, device, equipment and medium

Abstract

The present disclosure provides a text classification model training and text classification method, device, equipment and medium, which relate to the fields of deep learning and natural language processing. The specific implementation scheme is: clustering the obtained multiple sample texts to obtain at least one target cluster; generating a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs; using the text classification model to perform a first category prediction on each sample text to obtain a predicted label for each sample text; and performing a first training on the text classification model according to the predicted label and cluster label corresponding to each sample text. Since clustering can capture significant semantic features in sample texts, by clustering multiple sample texts to generate a cluster label corresponding to each sample text, and pre-training the text classification model based on the cluster label, the text classification model can effectively learn significant semantic information in the sample text before actual training, thereby improving model performance and performance.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, specifically to technical fields such as deep learning and natural language processing, and in particular to the training of text classification models and text classification methods, devices, equipment and media. Background Art

[0002] Natural Language Processing (NLP) is a key area of ​​research in computer science and artificial intelligence. Text classification is a fundamental aspect of NLP. Text classification uses text as an information carrier to organize and categorize text. It is widely used in various fields, such as digital libraries, public opinion analysis, news recommendations, and email filtering.

[0003] In order to achieve automatic text classification, when building a text classification model (also called a text classifier) ​​on existing sample texts, it is necessary to train the text classification model so as to classify the text to be classified based on the trained text classification model.

[0004] In order to improve the prediction effect of the model, it is very important to train the text classification model. Summary of the Invention

[0005] The present disclosure provides a text classification model training and text classification method, apparatus, device and medium.

[0006] According to one aspect of the present disclosure, a method for training a text classification model is provided, comprising:

[0007] Acquire multiple sample texts, and cluster the multiple sample texts to obtain at least one target cluster;

[0008] Generating a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs;

[0009] Using a text classification model to perform first category prediction on each of the sample texts to obtain a predicted label corresponding to each of the sample texts;

[0010] The text classification model is first trained according to the predicted labels and cluster labels corresponding to the sample texts.

[0011] According to another aspect of the present disclosure, a text classification method is provided, comprising:

[0012] Get the text to be classified;

[0013] The text to be classified is classified using a text classification model trained by the text classification model training method proposed in the above aspect of the present disclosure to obtain a classification label for the text to be classified.

[0014] According to another aspect of the present disclosure, a training device for a text classification model is provided, comprising:

[0015] An acquisition module, used to acquire multiple sample texts;

[0016] A clustering module, configured to cluster the plurality of sample texts to obtain at least one target cluster;

[0017] A generating module, configured to generate a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs;

[0018] A first prediction module is configured to perform a first category prediction on each of the sample texts using a text classification model to obtain a prediction label corresponding to each of the sample texts;

[0019] The second training module is used to perform a first training on the text classification model according to the prediction labels and cluster labels corresponding to each of the sample texts.

[0020] According to another aspect of the present disclosure, a text classification device is provided, comprising:

[0021] Acquisition module, used to obtain the text to be classified;

[0022] The classification module is used to classify the text to be classified using the text classification model trained by the text classification model training device proposed in the above-mentioned another aspect of the present disclosure, and obtain the classification label of the text to be classified.

[0023] According to another aspect of the present disclosure, there is provided an electronic device, including:

[0024] at least one processor; and

[0025] a memory communicatively connected to the at least one processor; wherein,

[0026] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method of the text classification model proposed in the above-mentioned first aspect of the present disclosure, or execute the text classification method proposed in the above-mentioned other aspect of the present disclosure.

[0027] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium of computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the training method of the text classification model proposed in the above-mentioned first aspect of the present disclosure, or to execute the text classification method proposed in the above-mentioned other aspect of the present disclosure.

[0028] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the training method of the text classification model proposed in the above-mentioned first aspect of the present disclosure, or implements the text classification method proposed in the above-mentioned other aspect of the present disclosure.

[0029] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0031] Figure 1 Schematic diagram of the training phase included in the text classification model;

[0032] Figure 2 A flowchart of a method for training a text classification model provided in the first embodiment of the present disclosure;

[0033] Figure 3 A flowchart of a text classification model training method provided in the second embodiment of the present disclosure;

[0034] Figure 4 A flowchart of a method for training a text classification model provided in the third embodiment of the present disclosure;

[0035] Figure 5 A flowchart of a method for training a text classification model provided in the fourth embodiment of the present disclosure;

[0036] Figure 6 A schematic diagram of the process of training a text classification model using the clustering stage and fine-tuning stage provided in the present disclosure;

[0037] Figure 7 A flowchart of a method for training a text classification model provided in the fifth embodiment of the present disclosure;

[0038] Figure 8 A schematic diagram of the process of training a text classification model using a three-stage training method provided in the present disclosure;

[0039] Figure 9A flowchart of a text classification method provided in Example 6 of the present disclosure;

[0040] Figure 10 A schematic diagram of the structure of a training device for a text classification model provided in Example 7 of the present disclosure;

[0041] Figure 11 This is a structural diagram of a text classification device provided in Example 8 of the present disclosure;

[0042] Figure 12 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0043] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0044] Text classification technology in natural language processing is widely used in various fields, but building text classification models requires labeling a large number of training samples. However, labeling training samples is time-consuming, hindering the rapid launch of the model. Furthermore, labeling training samples requires high labor costs, and the quality of the labeling varies widely.

[0045] In order to solve this type of problem, small sample technology can be used to train text classification models. By using a small number of training samples to train the text classification model, the model can have better learning and generalization capabilities to cope with the situation where the number of training samples is small in actual scenarios.

[0046] In related technologies, small sample technologies applied to text classification mainly focus on the following three areas:

[0047] The first direction is data. By explicitly introducing additional data for data enhancement, additional data can be used to assist in completing the optimization problem when searching in the search space, and prior knowledge can be used to enhance the supervision signal.

[0048] The second direction is the model, which uses prior knowledge to reduce the search space and shrink the size of the hypothesis space.

[0049] The third direction is algorithm, which introduces prior knowledge to optimize the search strategy algorithm, that is, to learn an optimal search strategy.

[0050] However, the above method has the following disadvantages:

[0051] 1. The quality reliability of data augmentation is low. In particular, when using similar data for training, it always introduces large noise.

[0052] 2. Model-based approaches often use pre-trained models for small sample transformation, but this approach does not provide good model prediction results during the fine-tuning phase.

[0053] As an example, the text classification model includes two training stages: pre-training stage and fine-tuning stage, such as Figure 1 As shown in the figure, in the pre-training stage, the effect of text classification can be improved by introducing prior knowledge, that is, some information can be learned in an unsupervised way in the pre-training stage, so that in the fine-tuning stage, a small amount of training text can be used to fine-tune the algorithm of the text classification model to achieve text classification.

[0054] 3. Algorithm-based methods mostly improve model parameters or search steps, and their application scope is relatively limited.

[0055] In response to the above problems, the present disclosure proposes a text classification model training and text classification method, device, equipment and medium.

[0056] The following describes the training of a text classification model and the text classification method, apparatus, device, and medium according to embodiments of the present disclosure with reference to the accompanying drawings.

[0057] Figure 2 A flowchart of the text classification model training method provided in the first embodiment of the present disclosure.

[0058] The embodiment of the present disclosure uses the example of configuring the training method of the text classification model in a training device for the text classification model. The training device for the text classification model can be applied to any electronic device so that the electronic device can perform the training function of the text classification model.

[0059] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, mobile terminal, server, etc. The mobile terminal can be, for example, a mobile phone, tablet computer, personal digital assistant, wearable device, etc., which are hardware devices with various operating systems, touch screens and / or display screens.

[0060] like Figure 2 As shown, the training method of the text classification model may include the following steps:

[0061] Step 201: Acquire multiple sample texts and cluster the multiple sample texts to obtain at least one target cluster.

[0062] In an embodiment of the present disclosure, multiple sample texts can be obtained, wherein the sample texts can be obtained from an existing training set, or the sample texts can be collected online, for example, sample texts can be collected online through web crawler technology, or the sample texts can be collected offline, for example, images of paper text content can be captured, and then each character in the image can be identified through OCR (Optical Character Recognition) technology to obtain sample text, etc. The present disclosure does not impose any restrictions on this.

[0063] The sample text may be an article, such as a novel, essay, or paper, or news, information, or a paragraph of text, etc. This disclosure does not impose any restrictions on this.

[0064] In an embodiment of the present disclosure, the obtained multiple sample texts may be clustered. For example, a clustering algorithm may be used to cluster the multiple sample texts to obtain at least one target cluster. The clustering algorithm may include a K-means clustering algorithm (K-means clustering algorithm), a KNN (K-Nearest Neighbor) classification algorithm, etc., which is not limited in the present disclosure.

[0065] The number of target clusters may be one or more, and this disclosure does not impose any limitation on this.

[0066] In a possible implementation of an embodiment of the present disclosure, in the process of clustering multiple sample texts, there may be abnormal sample texts. For example, the abnormal sample text does not belong to any cluster. At this time, the sample text can be eliminated so that the sample texts in the target cluster obtained by clustering are more concentrated, thereby using the target clusters with more concentrated sample texts to train the text classification model, so that the model can learn the stronger semantic features of each sample text in the same target cluster and ignore the interference caused by the abnormal sample text.

[0067] Step 202 : Generate a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs.

[0068] In an embodiment of the present disclosure, a cluster label can be used to indicate the cluster category to which the sample text belongs, wherein the cluster label can be a number in the form of a number, or a number in the form of an alphabet, or a number combining letters and numbers, or other character strings, special symbols, etc., and the present disclosure does not impose any restrictions on this.

[0069] In the embodiment of the present disclosure, a cluster label corresponding to each sample text can be generated based on the target cluster to which each sample text belongs, so as to indicate the cluster category to which each sample text belongs. The cluster labels corresponding to each sample text belonging to the same target cluster are the same.

[0070] For example, the number of target clusters obtained by clustering is 3, namely target cluster 1, target cluster 2, and target cluster 3. The cluster label corresponding to each sample text in target cluster 1 is 01, the cluster label corresponding to each sample text in target cluster 2 is 02, and the cluster label corresponding to each sample text in target cluster 3 is 03. In other words, cluster label 01 is used to indicate that the sample text belongs to target cluster 1, cluster label 02 is used to indicate that the sample text belongs to target cluster 2, and cluster label 03 is used to indicate that the sample text belongs to target cluster 3.

[0071] For another example, suppose that there are two target clusters obtained through clustering: target cluster 1 and target cluster 2. The cluster label corresponding to each sample text in target cluster 1 is a, and the cluster label corresponding to each sample text in target cluster 2 is b. In other words, cluster label a indicates that the sample text belongs to target cluster 1, and cluster label b indicates that the sample text belongs to target cluster 2.

[0072] As an example, semantically similar sample texts can be clustered into the same target cluster. For example, taking the sample texts as poems, poems about spring can be clustered into target cluster 1, poems about summer can be clustered into target cluster 2, poems about autumn can be clustered into target cluster 3, and poems about winter can be clustered into target cluster 4.

[0073] For example, the corresponding cluster label of each sample text in the generated target cluster 1 is 01, the corresponding cluster label of each sample text in the target cluster 2 is 02, the corresponding cluster label of each sample text in the target cluster 3 is 03, and the corresponding cluster label of each sample text in the target cluster 4 is 04.

[0074] It should be noted that the above examples of cluster labels are only exemplary. In actual application, the cluster label corresponding to each sample text can be set according to actual application requirements, and the present disclosure does not impose any restrictions on this.

[0075] Step 203: Use the text classification model to perform first category prediction on each sample text to obtain a prediction label corresponding to each sample text.

[0076] In the embodiment of the present disclosure, a text classification model may be used to perform first category prediction on each sample text, that is, the text classification model may be used to predict the cluster category to which each sample text belongs, so as to obtain a predicted label corresponding to each sample text.

[0077] For example, multiple sample texts are clustered to obtain target cluster 1 and target cluster 2, where the cluster label corresponding to target cluster 1 is 02, and the cluster label corresponding to target cluster 2 is 04. The text classification model is used to predict the cluster category of each sample text in target cluster 1, and the obtained predicted label can be 02. The classification model in this paper is used to predict the cluster category of each sample text 2 in target cluster 2, and the obtained predicted label can be 04.

[0078] Step 204 : Perform a first training on the text classification model according to the predicted labels and cluster labels corresponding to each sample text.

[0079] In the embodiment of the present disclosure, the text classification model may be first trained based on the predicted labels and cluster labels corresponding to each sample text.

[0080] As a possible implementation, for each sample text, when there is a difference between the predicted label and the cluster label corresponding to the sample text, it indicates that the prediction accuracy of the text classification model is not high. In this case, in order to improve the accuracy and reliability of the model prediction results and improve the model's prediction accuracy, the model parameters in the text classification model can be adjusted. That is, in the present disclosure, the text classification model can be first trained based on the difference between the predicted label and the cluster label corresponding to each sample text. That is, for each sample text, the model parameters in the text classification model can be adjusted based on the difference between the predicted label and the cluster label corresponding to the sample text.

[0081] For example, assuming that the cluster label corresponding to sample text 1 is 02 and the cluster label corresponding to sample text 2 is 04, the text classification model can be used to predict the cluster category of sample text 1 and sample text 2 to obtain the predicted label corresponding to each sample text. If the text classification model outputs a predicted label of 02 for sample text 1 and a predicted label of 04 for sample text 2, the model prediction is accurate and there is no need to adjust the model parameters. If the text classification model outputs a predicted label of 04 for sample text 1 and a predicted label of 04 for sample text 2, the model prediction is incorrect. In this case, the model parameters of the text classification model can be adjusted based on the difference between the cluster label and the predicted label corresponding to sample text 1.

[0082] As an example, a first loss function can be generated based on the difference between the predicted label and the cluster label corresponding to each sample text, wherein the value of the first loss function is positively correlated with the above-mentioned difference, that is, the smaller the difference, the smaller the value of the first loss function, and conversely, the greater the difference, the greater the value of the first loss function. Therefore, in the present disclosure, the text classification model can be first trained based on the value of the first loss function to minimize the value of the first loss function.

[0083] It should be noted that the above only uses the termination condition of the first training of the model as the minimization of the value of the first loss function as an example. In actual application, other termination conditions can also be set. For example, the termination condition can also be that the number of training times reaches a set threshold, the training time is greater than a set time threshold, etc. The present disclosure does not impose any restrictions on this.

[0084] The training method of the text classification model of the embodiment of the present disclosure obtains multiple sample texts and clusters the multiple sample texts to obtain at least one target cluster; generates a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs; uses the text classification model to perform a first category prediction on each sample text to obtain a prediction label corresponding to each sample text; and performs a first training on the text classification model according to the prediction label and cluster label corresponding to each sample text. In the present disclosure, since clustering can capture significant semantic features in the sample text, clustering multiple sample texts is performed to generate a cluster label corresponding to each sample text, and the text classification model is pre-trained based on the cluster label. This allows the text classification model to effectively learn significant semantic information in the sample text before actual training, thereby improving model performance and performance when the text classification model is actually trained using a small amount of sample text.

[0085] In order to clearly illustrate how multiple sample texts are clustered to obtain at least one target cluster in the above embodiments of the present disclosure, the present disclosure also proposes a training method for a text classification model.

[0086] Figure 3 This is a flowchart of the training method of the text classification model provided in the second embodiment of the present disclosure.

[0087] like Figure 3 As shown, the training method of the text classification model may include the following steps:

[0088] Step 301: Acquire multiple sample texts and the similarities between the multiple sample texts.

[0089] It should be noted that the above explanation of obtaining multiple sample texts is also applicable to this embodiment and will not be repeated here.

[0090] In the embodiment of the present disclosure, the similarity between the sample texts may be the text similarity between the sample texts, or may be the semantic similarity between the sample texts, which is not limited in the present disclosure.

[0091] In an embodiment of the present disclosure, after obtaining multiple sample texts, the similarity between the multiple sample texts can be obtained. For example, the similarity between the multiple sample texts can be calculated using algorithms such as the N-Gram model, cosine similarity, Pearson Correlation Coefficient, Euclidean Distance, and Manhattan Distance. The present disclosure does not impose any restrictions on this.

[0092] As a possible implementation method, feature extraction may be performed on each sample text to obtain a feature vector of each sample text, and the similarity between the sample texts may be calculated based on the feature vector of each sample text.

[0093] For example, assuming that sample text 1 is "Zhang San can sing and dance" and sample text 2 is "Li Si can dance and play the piano", first, obtain the subwords "Zhang San", "can", "sing", "and", and "dance" of sample text 1, and the subwords "Li Si", "can", "dance", "and", and "play the piano" of sample text 2; secondly, determine that the set composed of subwords appearing in sample text 1 and sample text 2 is {Zhang San, Li Si, can, sing, dance, and, play the piano}, then the feature vector corresponding to sample text 1 can be (1, 0, 1, 1, 1, 0), and the feature vector corresponding to sample text 2 can be (0, 1, 1, 0, 1, 1, 1), where each value in the feature vector corresponding to the sample text represents the number of times the subword in the corresponding position in the above set appears in the sample text. Finally, the similarity between sample text 1 and sample text 2 can be determined based on the similarity between the feature vector of sample text 1 and the feature vector of sample text 2.

[0094] It should be noted that, in actual applications, the feature vector corresponding to each sample text may also be determined according to other feature extraction algorithms, and the present disclosure does not impose any limitation on this.

[0095] As an example, the cosine similarity algorithm can be used to calculate the similarity between sample text 1 and sample text 2. The feature vector corresponding to sample text 1 is marked as (x1, x2, x3, x4, x5, x6, x7), and the feature vector corresponding to sample text 2 is marked as (y1, y2, y3, y4, y5, y6, y7). Then the cosine similarity S between sample text 1 and sample text 2 can be determined according to the following formula:

[0096]

[0097] The value range of S may be [-1, 1]. When S=1, it indicates that sample text 1 is completely similar to sample text 2; when S=-1, it indicates that sample text 1 is completely different from sample text 2.

[0098] As another example, the Euclidean distance algorithm can be used to calculate the similarity between sample text 1 and sample text 2. The feature vector corresponding to sample text 1 is marked as (x1, x2, x3, x4, x5, x6, x7), and the feature vector corresponding to sample text 2 is marked as (y1, y2, y3, y4, y5, y6, y7). Then the Euclidean distance d between sample text 1 and sample text 2 can be determined according to the following formula:

[0099]

[0100] It should be noted that when using distance measurement algorithms such as Euclidean distance and Manhattan distance to calculate the similarity between multiple sample texts, there is a negative relationship between the distance d and the similarity S. That is, the greater the distance, the smaller the similarity, and conversely, the smaller the distance, the greater the similarity.

[0101] Step 302: cluster the multiple sample texts according to the similarities between the multiple sample texts to obtain at least one target cluster.

[0102] In the embodiment of the present disclosure, multiple sample texts may be clustered according to the similarities between the multiple sample texts, thereby obtaining at least one target cluster.

[0103] As a possible implementation, a similarity threshold can be pre-set so that multiple sample texts can be clustered based on their similarity (e.g., textual similarity, semantic similarity) to obtain at least one target cluster. The similarity between sample texts belonging to the same target cluster is greater than the similarity threshold.

[0104] In a possible implementation of an embodiment of the present disclosure, in the process of clustering multiple sample texts, there may be abnormal sample texts. For example, the abnormal sample text does not belong to any cluster. At this time, the sample text can be eliminated so that the sample texts in the target cluster obtained by clustering are more concentrated, thereby using the target clusters with more concentrated sample texts to train the text classification model, so that the model can learn the stronger semantic features of each sample text in the same target cluster and ignore the interference caused by the abnormal sample text.

[0105] Step 303 : Generate a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs.

[0106] Step 304: Use the text classification model to perform first category prediction on each sample text to obtain a prediction label corresponding to each sample text.

[0107] Step 305 : Perform a first training on the text classification model according to the predicted labels and cluster labels corresponding to each sample text.

[0108] The execution process of steps 303 to 305 can refer to the execution process of any embodiment of the present disclosure and will not be described in detail here.

[0109] The text classification model training method of the disclosed embodiments obtains similarities between multiple sample texts and clusters the multiple sample texts based on the similarities to obtain at least one target cluster. Thus, clustering multiple sample texts based on similarities can improve the accuracy and reliability of the clustering results.

[0110] In order to clearly illustrate how, in any embodiment of the present disclosure, multiple sample texts are clustered according to the similarities between the multiple sample texts to obtain at least one target cluster, the present disclosure also proposes a training method for a text classification model.

[0111] Figure 4 This is a flowchart of the training method of the text classification model provided in Example 3 of the present disclosure.

[0112] like Figure 4 As shown, the training method of the text classification model may include the following steps:

[0113] Step 401: Acquire multiple sample texts and the similarities between the multiple sample texts.

[0114] The execution process of step 401 can refer to the execution process of any embodiment of the present disclosure, and will not be described in detail here.

[0115] Step 402 : Based on the set first number of clusters, a first clustering algorithm is used to cluster the plurality of sample texts according to the similarities between the plurality of sample texts to obtain initial clusters of the first number of clusters.

[0116] In the embodiment of the present disclosure, the first number of clusters may be preset.

[0117] In the embodiment of the present disclosure, the first clustering algorithm may be, for example, a K-means clustering algorithm, a KNN classification algorithm, etc., and the present disclosure does not impose any limitation on this.

[0118] In an embodiment of the present disclosure, a first clustering algorithm can be used according to a set first number of clusters to cluster multiple sample texts according to similarities between the sample texts (such as text similarity and semantic similarity), thereby obtaining initial clusters of the first number of clusters.

[0119] Step 403: Determine the distance between each initial cluster.

[0120] In the embodiment of the present disclosure, the distance between each initial cluster may be determined.

[0121] As an example, for each initial cluster, the reference sample text corresponding to the initial cluster can be determined from each sample text in the initial cluster, so that the distance between the reference sample texts of each initial cluster can be calculated, and the distance between the reference sample texts of each initial cluster can be used as the distance between each initial cluster.

[0122] For example, for any initial cluster, when determining the reference sample text corresponding to the initial cluster, a sample text can be randomly selected from the initial cluster, and the distance between the selected sample text and the remaining sample texts in the initial cluster other than the selected sample text is calculated, and the sum of the distances between the selected sample text and the above-mentioned remaining sample texts is calculated, and the sum of the distances is used as the weight of the selected sample text. Therefore, in the present disclosure, the reference sample text corresponding to the initial cluster can be determined from the sample texts in the initial cluster based on the weights corresponding to the sample texts in the initial cluster. For example, the sample text with the smallest weight can be used as the reference sample text of the initial cluster.

[0123] As another example, the cluster centers of the initial clusters may be determined, and the distances between the cluster centers of the initial clusters may be calculated, so that the distances between the cluster centers of the initial clusters may be used as the distances between the initial clusters.

[0124] For example, for any initial cluster, when determining the cluster center corresponding to the initial cluster, the mean of the feature vectors of each sample text in the initial cluster can be determined, and the mean of the feature vectors of each sample text in the initial cluster can be used as the cluster center of the initial cluster.

[0125] Step 404 : When the distance between the initial clusters is less than the set inter-cluster distance threshold, a second clustering algorithm is used to cluster the multiple sample texts according to the similarities between the multiple sample texts based on the inter-cluster distance threshold to obtain at least one target cluster.

[0126] In the embodiment of the present disclosure, the inter-cluster distance threshold may be preset.

[0127] In an embodiment of the present disclosure, the second clustering algorithm is different from the first clustering algorithm. For example, when the first clustering algorithm is the K-means clustering algorithm, the second clustering algorithm may be the KNN classification algorithm. When the first clustering algorithm is the KNN classification algorithm, the second clustering algorithm may be the K-means clustering algorithm, and so on. The present disclosure does not impose any restrictions on this.

[0128] In an embodiment of the present disclosure, after calculating the distances between the initial clusters, the distances between the initial clusters can be compared with a set inter-cluster distance threshold. If there is a situation where the distance between at least one initial cluster is less than the set inter-cluster distance threshold, or if the distances between all initial clusters are less than the set inter-cluster distance threshold, it indicates that the clustering effect of the first clustering algorithm is not good. At this time, in order to improve the accuracy and reliability of the clustering results, a second clustering algorithm can be selected according to the inter-cluster distance threshold, and the second clustering algorithm can be used to cluster the multiple sample texts according to the similarity between the multiple sample texts to obtain at least one target cluster.

[0129] The similarity between the sample texts belonging to the same target cluster is greater than a similarity threshold, and the distance between different target clusters is greater than or equal to a set inter-cluster distance threshold.

[0130] In a possible implementation of the embodiment of the present disclosure, when the distance between each initial cluster is less than a set inter-cluster distance threshold, the number of first clusters can also be adjusted (for example, the number of first clusters can be increased and / or the number of first clusters can be decreased), so that the first clustering algorithm can be used to cluster multiple sample texts again based on the adjusted number of first clusters and the inter-cluster distance threshold to obtain target clusters with the adjusted first number of clusters.

[0131] Thus, it is possible to cluster multiple sample texts in different ways to obtain at least one target cluster, which not only improves the clustering effect of the target cluster, but also improves the flexibility and applicability of the method.

[0132] Step 405 : Generate a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs.

[0133] Step 406: Use the text classification model to perform first category prediction on each sample text to obtain a prediction label corresponding to each sample text.

[0134] Step 407 : Perform a first training on the text classification model according to the predicted labels and cluster labels corresponding to each sample text.

[0135] The execution process of steps 405 to 407 can refer to the execution process of any embodiment of the present disclosure and will not be described in detail here.

[0136] The training method of the text classification model of the embodiment of the present disclosure is to cluster multiple sample texts according to the similarity between the multiple sample texts using a first clustering algorithm according to a set first number of clusters to obtain initial clusters of the first number of clusters; determine the distance between each initial cluster; and when the distance between each initial cluster is less than a set inter-cluster distance threshold, cluster the multiple sample texts according to the similarity between the multiple sample texts using a second clustering algorithm according to the inter-cluster distance threshold to obtain at least one target cluster. Therefore, when clustering the sample texts using the first clustering algorithm, if the clustering effect of the first clustering algorithm is not good, another clustering algorithm can be replaced to cluster the multiple sample texts again, which can improve the clustering effect of the target cluster.

[0137] In a possible implementation of the embodiment of the present disclosure, after the text classification model is first trained, in order to improve the classification effect of the text classification model, the text classification model may also be second trained. Figure 5 , the second training process is described in detail.

[0138] Figure 5 This is a flowchart of the training method of the text classification model provided in the fourth embodiment of the present disclosure.

[0139] like Figure 5 As shown, the training method of the text classification model may include the following steps:

[0140] Step 501: Acquire multiple sample texts and cluster the multiple sample texts to obtain at least one target cluster.

[0141] Step 502 : Generate a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs.

[0142] Step 503: Use the text classification model to perform first category prediction on each sample text to obtain a prediction label corresponding to each sample text.

[0143] Step 504 : Perform a first training on the text classification model according to the predicted labels and cluster labels corresponding to each sample text.

[0144] The execution process of steps 501 to 504 can refer to the execution process of any embodiment of the present disclosure and will not be described in detail here.

[0145] Step 505 : Use the first trained text classification model to perform second category prediction on each sample text to obtain a classification label.

[0146] In the embodiment of the present disclosure, the classification labels may be different for different text classification tasks, wherein the text classification tasks may include sentiment classification, whether classification, other classifications, etc.

[0147] As an example, let's take sentiment classification as an example. The goal of sentiment classification is to determine the sentiment polarity (positive or negative) of a sample text or the sentiment polarity of a target entity within the sample text. When the goal of sentiment classification is to determine the sentiment polarity of a sample text or the sentiment polarity of a target entity within the sample text, the classification label (i.e., sentiment polarity) can be "positive" or "negative."

[0148] As another example, let's take the text classification task of yes / no classification. The goal of yes / no classification is to give an answer to a yes / no question. For example, if the yes / no question is "Did you go to the dance yesterday?", the classification label (i.e., the answer) can include three categories: "yes," "no," or "uncertain."

[0149] It should be noted that when the text classification task is other classifications, for other sentence classification tasks, opinion classification tasks, entity classification tasks, etc., a similar method as described above can be used to determine the classification label corresponding to the sample text.

[0150] In the embodiment of the present disclosure, the first trained text classification model can be used to perform a second category prediction on each sample text, thereby obtaining a classification label corresponding to each sample text. That is, in the present disclosure, the first trained text classification model can be used to perform text classification on each sample text, thereby obtaining a classification label for each sample text.

[0151] Step 506 : performing a second training on the text classification model after the first training according to the difference between the classification label corresponding to each sample text and the annotated true label.

[0152] In the embodiment of the present disclosure, each sample text may be marked with a true label, wherein the true label is used to indicate the classification category to which the corresponding sample text belongs.

[0153] In the disclosed embodiments, when there is a discrepancy between the classification label corresponding to each sample text and the annotated true label, it indicates that the model's prediction accuracy is low. Therefore, to improve the model's prediction accuracy, that is, to improve the accuracy and reliability of the model's classification results, the text classification model that has undergone the first training can be subjected to a second training. Specifically, for each sample text, the model parameters in the text classification model can be adjusted based on the discrepancy between the classification label corresponding to the sample text and the true label.

[0154] As an example, a second loss function can be generated based on the difference between the classification label corresponding to each sample text and the true label, wherein the value of the second loss function is positively correlated with the above-mentioned difference, that is, the smaller the difference, the smaller the value of the second loss function, and conversely, the greater the difference, the greater the value of the second loss function. Therefore, in the present disclosure, the text classification model can be trained for the second time based on the value of the second loss function to minimize the value of the second loss function.

[0155] It should be noted that the above only uses the termination condition of the second training of the model as an example of minimizing the value of the second loss function. In actual application, other termination conditions can also be set. For example, the termination condition can also be that the number of training times reaches a set threshold, the training time is greater than a set threshold, etc. The present disclosure does not impose any restrictions on this.

[0156] As an example, taking the text classification task as sentiment classification, the second training process of the text classification model suitable for the sentiment classification task is exemplified by the following example:

[0157] When the purpose of sentiment classification is to determine the sentiment polarity of a sample text, the classification label (i.e., sentiment polarity) can be "positive" or "negative." Suppose the sample text is "I'm very happy to have won a prize in today's competition," and the true label is "positive." If the text classification model after the first training is used to perform a second category prediction (i.e., text classification) on the sample text, and the resulting classification label is "positive," then the model prediction is correct and there is no need to adjust the model parameters. If the text classification model after the first training is used to perform a second category prediction on the sample text, and the resulting classification label is "negative," then the model prediction is incorrect, and the model parameters can be adjusted.

[0158] When the purpose of sentiment classification is to determine the sentiment polarity of the target entity in the sample data, the classification label (i.e., sentiment polarity) can be "positive" or "negative." Suppose the sample text is "The environment in Region A is better this year," the target entity is "Region A," and the annotated true label is "positive." If the text classification model after the first training is used to predict the second category of the sample text, and the resulting classification label is "positive," then the model prediction is correct and there is no need to adjust the model parameters. If the text classification model after the first training is used to predict the second category of the sample text, and the resulting classification label is "negative," then the model prediction is incorrect, and the model parameters can be adjusted.

[0159] As another example, taking the text classification task as whether to classify or not, the second training process of the text classification model applicable to the whether to classify or not task is exemplified by the following example:

[0160] Suppose the yes / no question is "Did you attend the dance yesterday?", the sample text is "Zhang San attended the dance on time yesterday", and the annotated true label is "yes". If the text classification model after the first training is used to predict the second category of the sample text, the obtained classification label is "yes", then the model prediction is correct and there is no need to adjust the model parameters; if the text classification model after the first training is used to predict the second category of the sample text, the obtained classification label is "no" or "uncertain", then the model prediction is wrong, and the model parameters can be adjusted at this time.

[0161] It should be noted that when the text classification task is other classifications, for other sentence classification tasks, opinion classification tasks, entity classification tasks, etc., a similar method to the above can be used to determine the classification label corresponding to the sample text, so as to determine whether there is a difference between the classification label and the true label corresponding to the sample text. If so, the text classification model that has undergone the first training can be trained for the second time based on the above difference. I will not go into details here.

[0162] As an example, the first training process of the text classification model can be called the clustering stage, and the second training process can be called the fine-tuning stage, such as Figure 6 As shown, in the clustering stage, the text classification model can make a first category prediction for the sample text through the cluster label classification layer to obtain the predicted label corresponding to the sample text, so that the text classification model can be first trained based on the difference between the cluster label and the predicted label corresponding to the sample text.

[0163] For example, an unsupervised clustering phase can be performed using each sample text, and a clustering algorithm can be used to cluster each sample text to obtain target clusters. Cluster labels can be added to each sample text according to the target cluster to which each sample text belongs, and then the cluster labels of the clusters can be learned using a model so that the model can learn some significant semantic information.

[0164] In the fine-tuning stage, the text classification model can make a second category prediction for the sample text through the true label classification layer to obtain the classification label corresponding to the sample text, so that the text classification model can be trained for the second time based on the difference between the classification label of the sample text and the annotated true label.

[0165] It can be understood that clustering can capture the more significant semantic features in the sample text. Clustering can be used to obtain cluster labels (or auxiliary labels) of the sample text. Using the above cluster labels to train the text classification model can enable the text classification model to learn the significant semantic information in the sample text before actual training, thereby improving the performance of the model when a small amount of sample text is used to conduct actual training on the text classification model (i.e., the second training).

[0166] The text classification model training method of the disclosed embodiments uses a first-trained text classification model to perform a second category prediction on each sample text to obtain a classification label. Based on the difference between the classification label corresponding to each sample text and the annotated true label, the first-trained text classification model is subjected to a second training. Thus, by using the true labels annotated with multiple sample texts to perform a second training on the first-trained text classification model, the classification performance of the text classification model can be improved, that is, the accuracy and reliability of the text classification results can be improved.

[0167] In a possible implementation of the embodiment of the present disclosure, before the first training of the text classification model, the text classification model may be further trained to further improve the performance and performance of the model. Figure 7 , the third training process is described in detail.

[0168] Figure 7 This is a flowchart of the training method of the text classification model provided in Example 5 of the present disclosure.

[0169] like Figure 7 As shown, the training method of the text classification model may include the following steps:

[0170] Step 701: Acquire multiple sample texts.

[0171] The execution process of step 701 can refer to the execution process of any embodiment of the present disclosure and will not be described in detail here.

[0172] Step 702 : for any sample text among the plurality of sample texts, mask at least one sample character in any sample text to obtain a masked sample text.

[0173] In the embodiment of the present disclosure, masking the sample characters means replacing the sample characters with mask characters, wherein the mask characters can be preset fixed characters, or the mask characters can also be random characters, which is not limited by the present disclosure.

[0174] In an embodiment of the present disclosure, a mask character may be used to mask at least one sample character in any of a plurality of sample texts, thereby obtaining a masked sample text. The number of sample characters masked may be one or more, and the present disclosure does not impose any restrictions thereon. The number of sample texts masked may be, but is not limited to, one, and the present disclosure does not impose any restrictions thereon either.

[0175] As an example, taking the masking process of the sample text "I'm very happy to have won a prize in the competition today" as an example, assuming that the sample character to be masked is "competition" and the masking character is "high jump", then the masked sample text can be "I'm very happy to have won a prize in the high jump today".

[0176] It should be noted that the above example only uses the masking character "high jump" for illustrative purposes. Those skilled in the art can, according to the actual business needs in practical applications, select the masking character used for masking the sample characters. That is, the embodiments of the present disclosure do not make specific limitations on the masking character.

[0177] Step 703, use a text classification model to perform character prediction on the masked sample text to obtain a predicted text.

[0178] In the embodiments of the present disclosure, a text classification model can be used to perform character prediction on the masked sample text, thereby obtaining a predicted text. That is to say, in the present disclosure, the text classification model can predict all characters in the entire text in a manner similar to machine translation to obtain a predicted text.

[0179] Still taking the above example, "I'm very happy to have won a prize in the high jump today" can be input into the text classification model. The text classification model performs character prediction on the entire text "I'm very happy to have won a prize in the high jump today". The predicted text output by the text classification model can be "I'm very happy to have won a prize in the competition today" and / or "I'm very happy to have won a prize in the high jump today".

[0180] Step 704, perform the third training on the text classification model according to the difference between any sample text and the corresponding predicted text.

[0181] In the embodiments of the present disclosure, it can be determined whether there is a difference between the predicted text output by the text classification model and any sample text. When there is a difference between the any sample text and the corresponding predicted text, it indicates that the prediction accuracy of the model is not high. Therefore, in order to improve the prediction accuracy of the model, the model parameters in the text classification model can be adjusted, that is, the third training can be performed on the text classification model.

[0182] As an example, the third loss function can be generated according to the difference between any sample text and the corresponding predicted text. Among them, the value of the third loss function has a positive relationship with the above difference, that is, the smaller the difference, the smaller the value of the third loss function, and vice versa, the larger the difference, the larger the value of the third loss function. Thus, in the present disclosure, the third training can be performed on the text classification model according to the value of the third loss function to minimize the value of the third loss function.

[0183] It should be noted that the above only uses the termination condition of the third training of the model as an example of minimizing the value of the third loss function. In actual application, other termination conditions can also be set. For example, the termination condition can also be that the number of training times reaches a set threshold, the training time is greater than a set time threshold, etc. The present disclosure does not impose any restrictions on this.

[0184] In a possible implementation of the embodiment of the present disclosure, a text classification model can also be used to predict the masked characters in the masked sample text to obtain at least one predicted character, so that the text classification model can be trained for the third time based on the difference between at least one sample character and at least one predicted character.

[0185] In this way, the text classification model can be trained for the third time in different ways, which can improve the flexibility and applicability of the method.

[0186] In the disclosed embodiments, the masked sample text can be input into a text classification model, which then predicts the masked characters in the masked sample text to obtain at least one predicted character. In other words, in the disclosed embodiments, the text classification model can only predict the masked characters, similar to a cloze test task.

[0187] The number of predicted characters is the same as the number of masked sample characters.

[0188] Therefore, in the present disclosure, the text classification model can be subjected to third training according to the difference between at least one sample character and at least one predicted character, that is, the model parameters in the text classification model can be adjusted.

[0189] As an example, a fourth loss function can be generated based on the difference between at least one sample character and at least one predicted character, wherein the value of the fourth loss function is positively correlated with the above-mentioned difference, that is, the smaller the difference, the smaller the value of the fourth loss function, and conversely, the greater the difference, the greater the value of the fourth loss function. Therefore, in the present disclosure, the text classification model can be subjected to a third training based on the value of the fourth loss function to minimize the value of the fourth loss function.

[0190] It should be noted that the above only uses the termination condition of the third training of the model as an example of minimizing the value of the fourth loss function. In actual application, other termination conditions can also be set. For example, the termination condition can also be that the number of training times reaches a set threshold, the training time is greater than a set time threshold, etc. The present disclosure does not impose any restrictions on this.

[0191] Taking the above example, the text "Today, Bi Gao won an award and is very happy" can be input into the text classification model. The text classification model predicts the masked character "competition". If the predicted character output by the text classification model is "competition", the model prediction is accurate and no adjustment is needed for the model parameters. If the predicted character output by the text classification model is other characters, it is determined that the model prediction is incorrect and the model parameters need to be adjusted, that is, the model is subjected to the third training.

[0192] It can be understood that in addition to masking the sample text and pre-training the text classification model based on the masked sample text, an autoregressive method can also be used to pre-train the text classification model, that is, based on the existing knowledge of the sample text, the sample text is filled or expanded. The present disclosure does not limit this.

[0193] Step 705: Cluster multiple sample texts to obtain at least one target cluster.

[0194] Step 706: Generate a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, where the cluster label is used to indicate the cluster category to which the sample text belongs.

[0195] The execution process of steps 705 to 706 can be referred to the execution process of any embodiment of the present disclosure and will not be elaborated here.

[0196] Step 707: Use the text classification model after the third training to perform the first category prediction on each sample text to obtain a prediction label corresponding to each sample text.

[0197] In the embodiment of the present disclosure, the text classification model after the third training can be used to perform the first category prediction on each sample text to obtain a prediction label corresponding to each sample text.

[0198] Thus, before the first training of the text classification model, the third training of the text classification model is performed first, which can improve the performance and performance of the model, thereby improving the training effect of the model.

[0199] Step 708: Perform the first training on the text classification model according to the prediction label and the cluster label corresponding to each sample text.

[0200] The execution process of step 708 can be referred to the execution process of any embodiment of the present disclosure and will not be elaborated here.

[0201] The training method of the text classification model of the embodiment of the present disclosure is as follows: for any sample text among multiple sample texts, at least one sample character in any sample text is masked to obtain the masked sample text; the text classification model is used to perform character prediction on the masked sample text to obtain the predicted text; the text classification model is trained for a third time based on the difference between any sample text and the corresponding predicted text; the text classification model that has undergone the third training is used to perform first category prediction on each sample text to obtain the predicted label corresponding to each sample text. Thus, by processing the sample text in a masked manner and pre-training the text classification model based on the masked sample text, the performance and performance of the model can be further improved, thereby improving the prediction effect of the model.

[0202] As an example, unlike Figure 1 The existing two-stage training method (i.e., pre-training stage and fine-tuning stage) is used to train the text classification model. In the present disclosure, an intermediate stage, i.e., clustering stage, can be added between the pre-training stage and the fine-tuning stage. Figure 8 As shown, the present disclosure can train the text classification model through a three-stage training method (i.e., pre-training stage, clustering stage and fine-tuning stage), wherein the pre-training stage is the third training of the text classification model, the clustering stage is the first training of the text classification model, and the fine-tuning stage is the second training of the text classification model.

[0203] The clustering phase doesn't require consideration of task scenarios or categories, meaning there's no need to design specifically for those scenarios or categories. This makes it a universal, simple, and effective training method. By first training the model, it learns the correspondence between text and cluster labels—that is, the correspondence between text and the cluster category to which it belongs. This allows the model to learn stronger semantic features and ignore the interference caused by outliers.

[0204] After using the model to learn the cluster category to which the text belongs, in the fine-tuning stage, only the semantic feature capabilities learned by the model itself are needed. Therefore, in the actual training of the fine-tuning stage, there is no need to use Figure 6 The cluster label classification layer in the clustering stage shown in the figure only needs to use the real labels of the sample text to learn the new classification layer (i.e. Figure 6 The true label classification layer in the clustering stage, where the cluster label classification layer learned in the clustering stage and the true label classification layer learned in the fine-tuning stage can be as follows Figure 6 shown.

[0205] In summary, the three-stage training method for training text classification models has the following advantages: no data augmentation of sample texts, no need to use additional mutual data, and no noise is introduced; when a small amount of sample text is used for the second training of the model, the text classification effect of the model can be improved; no special design is required for scenarios, data or categories, the method is simple and convenient, and has strong versatility.

[0206] The above are various embodiments corresponding to the training method of the text classification model. The present disclosure also proposes an application method of the text classification model, namely, a text classification method.

[0207] Figure 9 This is a flowchart of the text classification method provided in Example 6 of the present disclosure.

[0208] like Figure 9 As shown, the text classification method may include the following steps:

[0209] Step 901: Obtain the text to be classified.

[0210] In the embodiments of the present disclosure, the text to be classified can be obtained from an existing test set, or the text to be classified can also be collected online, for example, by using web crawler technology, or the text to be classified can also be collected offline, or the text to be classified can also be text entered by the user, etc. The embodiments of the present disclosure do not limit this.

[0211] Step 902: Classify the text to be classified using the trained text classification model to obtain a classification label for the text to be classified.

[0212] The text classification model may be trained using any of the above method embodiments.

[0213] In an embodiment of the present disclosure, the text to be classified may be input into a trained text classification model, and the text classification model may classify the text to be classified to obtain a classification label of the text to be classified output by the text classification model.

[0214] The text classification method of the disclosed embodiment obtains a text to be classified and classifies it using a trained text classification model to obtain a classification label for the text to be classified. Thus, based on deep learning technology, the classification of the text to be classified can improve the accuracy and reliability of the classification results.

[0215] With the above Figures 2 to 7 Corresponding to the training method of the text classification model provided in the embodiment, the present disclosure also provides a training device for a text classification model. Figures 2 to 7The training method of the text classification model provided in the embodiment corresponds to the embodiment, so the implementation method of the text classification model training method is also applicable to the training device of the text classification model provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0216] Figure 10 This is a structural diagram of the training device for the text classification model provided in Example 7 of the present disclosure.

[0217] like Figure 10 As shown, the training device 1000 for the text classification model may include: an acquisition module 1001 , a clustering module 1002 , a generation module 1003 , a first prediction module 1004 and a first training module 1005 .

[0218] The acquisition module 1001 is used to acquire multiple sample texts.

[0219] The clustering module 1002 is configured to cluster multiple sample texts to obtain at least one target cluster.

[0220] The generating module 1003 is configured to generate a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs.

[0221] The first prediction module 1004 is configured to perform a first category prediction on each sample text using a text classification model to obtain a prediction label corresponding to each sample text.

[0222] The first training module 1005 is used to perform a first training on the text classification model according to the predicted labels and cluster labels corresponding to each sample text.

[0223] In a possible implementation of the embodiment of the present disclosure, the clustering module 1002 is specifically configured to:

[0224] The similarities between the multiple sample texts are obtained; and the multiple sample texts are clustered according to the similarities between the multiple sample texts to obtain at least one target cluster.

[0225] In a possible implementation of the embodiment of the present disclosure, the clustering module 1002 is specifically configured to: cluster the plurality of sample texts according to the similarities between the plurality of sample texts, based on a set first number of clusters, using a first clustering algorithm to obtain initial clusters of the first number of clusters; determine the distance between each initial cluster; and, when the distance between each initial cluster is less than a set inter-cluster distance threshold, cluster the plurality of sample texts according to the similarities between the plurality of sample texts, based on the inter-cluster distance threshold, using a second clustering algorithm to obtain at least one target cluster, based on the similarities between the plurality of sample texts.

[0226] In a possible implementation of the embodiment of the present disclosure, the clustering module 1002 is specifically configured to: cluster the plurality of sample texts according to the similarities between the plurality of sample texts according to a set first number of clusters, so as to obtain initial clusters of the first number of clusters; determine the distance between each initial cluster; adjust the first number of clusters when the distance between each initial cluster is less than a set inter-cluster distance threshold; and cluster the plurality of sample texts again according to the adjusted first number of clusters and the inter-cluster distance threshold, using the first clustering algorithm to obtain target clusters of the adjusted first number of clusters.

[0227] In a possible implementation of the embodiment of the present disclosure, the text classification model training device 1000 may further include:

[0228] The second prediction module is used to use the text classification model after the first training to perform a second category prediction on each sample text to obtain a classification label.

[0229] The second training module is used to perform a second training on the text classification model after the first training according to the difference between the classification label corresponding to each sample text and the marked true label.

[0230] In a possible implementation of the embodiment of the present disclosure, the text classification model training device 1000 may further include:

[0231] The first masking module is configured to mask at least one sample character in any sample text among a plurality of sample texts to obtain a masked sample text.

[0232] The third prediction module is used to use a text classification model to perform character prediction on the masked sample text to obtain a predicted text.

[0233] The third training module is used to perform third training on the text classification model according to the difference between any sample text and the corresponding predicted text.

[0234] In a possible implementation of the embodiment of the present disclosure, the text classification model training device 1000 may further include:

[0235] The second masking module is configured to mask at least one sample character in any sample text among the plurality of sample texts to obtain a masked sample text.

[0236] The fourth prediction module is configured to predict the masked characters in the masked sample text using a text classification model to obtain at least one predicted character.

[0237] The fourth training module is used to perform a third training on the text classification model according to the difference between the at least one sample character and the at least one predicted character.

[0238] In a possible implementation of the embodiment of the present disclosure, the first prediction module 1004 is specifically configured to: use the third trained text classification model to perform first category prediction on each sample text to obtain a prediction label corresponding to each sample text.

[0239] The training device of the text classification model of the embodiment of the present disclosure obtains multiple sample texts and clusters the multiple sample texts to obtain at least one target cluster; generates a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs; uses the text classification model to perform a first category prediction on each sample text to obtain a prediction label corresponding to each sample text; and performs a first training on the text classification model according to the prediction label and cluster label corresponding to each sample text. In the present disclosure, since clustering can capture significant semantic features in the sample text, clustering multiple sample texts is performed to generate a cluster label corresponding to each sample text, and the text classification model is pre-trained based on the cluster label. This allows the text classification model to effectively learn significant semantic information in the sample text before actual training, thereby improving model performance and performance when the text classification model is actually trained using a small amount of sample text.

[0240] With the above Figure 9 Corresponding to the text classification method provided in the embodiment, the present disclosure also provides a text classification device. Since the text classification device provided in the embodiment of the present disclosure is consistent with the above Figure 9 The text classification method provided in the embodiment corresponds to the text classification method, so the implementation of the text classification method is also applicable to the text classification device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0241] Figure 11 This is a structural diagram of the text classification device provided in Example 8 of the present disclosure.

[0242] like Figure 11 As shown, the text classification device 1100 may include: an acquisition module 1101 and a classification module 1102 .

[0243] The acquisition module 1101 is used to acquire the text to be classified.

[0244] Classification module 1102, for adopting Figure 10 The text classification model trained by the text classification model training device shown classifies the text to be classified to obtain the classification label of the text to be classified.

[0245] The text classification device of the disclosed embodiment obtains a text to be classified and classifies it using a trained text classification model to obtain a classification label for the text to be classified. Thus, based on deep learning technology, the classification of the text to be classified can improve the accuracy and reliability of the classification results.

[0246] In order to implement the above embodiments, the present disclosure also provides an electronic device, which may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the text classification model training method or text classification method proposed in any of the above embodiments of the present disclosure.

[0247] In order to implement the above embodiments, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the text classification model training method or text classification method proposed in any of the above embodiments of the present disclosure.

[0248] In order to implement the above embodiments, the present disclosure further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the text classification model training method or text classification method proposed in any of the above embodiments of the present disclosure.

[0249] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0250] Figure 12 A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0251] like Figure 12As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1202 or a computer program loaded from a storage unit 1208 into a RAM (Random Access Memory) 1203. Various programs and data required for the operation of the electronic device 1200 can also be stored in the RAM 1203. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An I / O (Input / Output) interface 1205 is also connected to the bus 1204.

[0252] Multiple components in the electronic device 1200 are connected to the I / O interface 1205, including an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a magnetic disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the electronic device 1200 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0253] Computing unit 1201 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of computing unit 1201 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphic Processing Units), various specialized AI (Artificial Intelligence) computing chips, various computing units that run machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processors, controllers, microcontrollers, etc. Computing unit 1201 performs the various methods and processes described above, such as the aforementioned text classification model training method or text classification method. For example, in some embodiments, the aforementioned text classification model training method or text classification method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by computing unit 1201, one or more steps of the above-described text classification model training method or text classification method can be performed. Alternatively, in other embodiments, computing unit 1201 can be configured to perform the above-described text classification model training method or text classification method in any other appropriate manner (e.g., via firmware).

[0254] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System on Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0255] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0256] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0257] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0258] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0259] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship is established by computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and poor scalability of traditional physical hosts and VPS services. The server may also be a server in a distributed system or a server integrated with blockchain.

[0260] It's important to note that artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). This encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0261] According to the technical solution of the embodiment of the present disclosure, a plurality of sample texts are obtained and clustered to obtain at least one target cluster; a cluster label corresponding to each sample text is generated according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs; a text classification model is used to perform a first category prediction on each sample text to obtain a prediction label corresponding to each sample text; and a first training of the text classification model is performed according to the prediction label and cluster label corresponding to each sample text. In the present disclosure, since clustering can capture significant semantic features in the sample text, clustering a plurality of sample texts is performed to generate a cluster label corresponding to each sample text, and the text classification model is pre-trained based on the cluster label. This allows the text classification model to effectively learn significant semantic information in the sample text before actual training, thereby improving model performance and performance when the text classification model is actually trained using a small amount of sample text.

[0262] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions proposed in this disclosure can be achieved. This is not limited herein.

[0263] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for training a text classification model, the method comprising: Acquire multiple sample texts, mask at least one sample character in any sample text from the multiple sample texts to obtain the masked sample text, and perform a third training on the text classification model using the masked sample text; Clustering the multiple sample texts to obtain at least one target cluster; Generating a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs; Using the third trained text classification model to perform first category prediction on each of the sample texts to obtain a predicted label corresponding to each of the sample texts; Performing a first training on the text classification model according to the predicted labels and cluster labels corresponding to each of the sample texts; Using the first trained text classification model to perform a second category prediction on each of the sample texts to obtain a classification label; According to the difference between the classification label corresponding to each sample text and the marked true label, the text classification model after the first training is subjected to a second training.

2. The method according to claim 1, wherein Clustering the plurality of sample texts to obtain at least one target cluster includes: Obtaining similarities between the multiple sample texts; The plurality of sample texts are clustered according to similarities between the plurality of sample texts to obtain at least one target cluster.

3. The method according to claim 2, wherein: Clustering the plurality of sample texts according to the similarities between the plurality of sample texts to obtain at least one target cluster includes: According to the set first number of clusters, clustering the plurality of sample texts according to similarities between the plurality of sample texts using a first clustering algorithm to obtain initial clusters of the first number of clusters; Determining the distance between each of the initial clusters; When the distance between the initial clusters is less than a set inter-cluster distance threshold, a second clustering algorithm is used to cluster the multiple sample texts according to the similarities between the multiple sample texts based on the inter-cluster distance threshold to obtain at least one target cluster.

4. The method according to claim 2, wherein: Clustering the plurality of sample texts according to the similarities between the plurality of sample texts to obtain at least one target cluster includes: According to the set first number of clusters, clustering the plurality of sample texts according to similarities between the plurality of sample texts using a first clustering algorithm to obtain initial clusters of the first number of clusters; Determining the distance between each of the initial clusters; When the distance between each of the initial clusters is less than a set inter-cluster distance threshold, adjusting the number of the first clusters; The first clustering algorithm is used to cluster the plurality of sample texts again according to the adjusted first number of clusters and the inter-cluster distance threshold, so as to obtain target clusters having the adjusted first number of clusters.

5. The method according to claim 1, wherein The third training of the text classification model using the masked sample text includes: Using the text classification model to perform character prediction on the masked sample text to obtain predicted text; The text classification model is subjected to a third training according to the difference between any sample text and the corresponding predicted text.

6. The method according to claim 1, wherein The third training of the text classification model using the masked sample text includes: Using the text classification model to predict the masked characters in the masked sample text to obtain at least one predicted character; The text classification model is subjected to a third training according to the difference between the at least one sample character and the at least one predicted character.

7. A text classification method, comprising: Get the text to be classified; The text to be classified is classified using a text classification model trained by the method according to any one of claims 1 to 6 to obtain a classification label for the text to be classified.

8. A training device for a text classification model, comprising: An acquisition module, used to acquire multiple sample texts; A clustering module, configured to cluster the plurality of sample texts to obtain at least one target cluster; A generating module, configured to generate a cluster label corresponding to each sample text according to the target cluster to which each sample text belongs, wherein the cluster label is used to indicate the cluster category to which the sample text belongs; a first prediction module, configured to use a third trained text classification model to perform a first category prediction on each of the sample texts, and obtain a prediction label corresponding to each of the sample texts; A first training module, configured to perform a first training on the text classification model according to the prediction labels and cluster labels corresponding to the sample texts; The device further comprises: A second prediction module is used to use the first trained text classification model to perform a second category prediction on each sample text to obtain a classification label; A second training module is used to perform a second training on the text classification model after the first training according to the difference between the classification label corresponding to each of the sample texts and the marked true label; The device is also used for: For any sample text among the multiple sample texts, at least one sample character in the any sample text is masked to obtain a masked sample text, and the masked sample text is used to perform the third training on the text classification model.

9. The device according to claim 8, wherein The clustering module is specifically used to: Obtaining similarities between the multiple sample texts; The plurality of sample texts are clustered according to similarities between the plurality of sample texts to obtain at least one target cluster.

10. The device according to claim 9, wherein The clustering module is specifically used to: According to the set first number of clusters, clustering the plurality of sample texts according to similarities between the plurality of sample texts using a first clustering algorithm to obtain initial clusters of the first number of clusters; Determining the distance between each of the initial clusters; When the distance between the initial clusters is less than a set inter-cluster distance threshold, a second clustering algorithm is used to cluster the multiple sample texts according to the similarities between the multiple sample texts based on the inter-cluster distance threshold to obtain at least one target cluster.

11. The device according to claim 9, wherein The clustering module is specifically used to: According to the set first number of clusters, clustering the plurality of sample texts according to similarities between the plurality of sample texts using a first clustering algorithm to obtain initial clusters of the first number of clusters; Determining the distance between each of the initial clusters; When the distance between each of the initial clusters is less than a set inter-cluster distance threshold, adjusting the number of the first clusters; The first clustering algorithm is used to cluster the plurality of sample texts again according to the adjusted first number of clusters and the inter-cluster distance threshold, so as to obtain target clusters having the adjusted first number of clusters.

12. The device according to claim 8, wherein The device further comprises: a third prediction module, configured to perform character prediction on the masked sample text using the text classification model to obtain a predicted text; The third training module is used to perform third training on the text classification model according to the difference between any sample text and the corresponding predicted text.

13. The device according to claim 8, wherein The device further comprises: a fourth prediction module, configured to predict the masked characters in the masked sample text using the text classification model to obtain at least one predicted character; The fourth training module is configured to perform a third training on the text classification model according to the difference between the at least one sample character and the at least one predicted character.

14. A text classification device, comprising: Acquisition module, used to obtain the text to be classified; A classification module is used to classify the text to be classified using a text classification model trained by the device according to any one of claims 8 to 13 to obtain a classification label for the text to be classified.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6, or, alternatively, to perform the method of claim 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 6, or to execute the method according to claim 7.

17. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 6, or implements the steps of the method according to claim 7.

Images