Unsupervised text classification method and device, equipment, storage medium and product

Abstract

The invention provides an unsupervised text classification method and device, equipment, a storage medium and a product, and relates to the technical field of artificial intelligence. Dividing the to-be-classified text set into a to-be-labeled sample set and a to-be-labeled text set; based on the large language model, performing classification labeling on the to-be-labeled sample set to obtain a labeled sample set; based on the labeled sample set, training the initial text classifier to obtain a text classifier; and based on the text classifier, carrying out classification labeling on the to-be-labeled text set to obtain a text classification result. Through the mode, the problems of low classification efficiency and classification accuracy caused by the phenomena of complexity in similarity calculation, difficulty in clustering setting, poor transfer learning effect and the like can be avoided, and the text classification efficiency and the accuracy of a text classification result can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an unsupervised text classification method, device, equipment, storage medium and product. Background Art

[0002] As a natural language processing task, text classification plays an important role in scientific and technological literature processing and services. Text classification often requires large-scale, widely representative annotated data as training corpus, but the selection and annotation of training corpus are difficult, which will affect the execution of text classification tasks. Therefore, there is a strong demand for unsupervised text classification.

[0003] In some related technologies, there are three unsupervised text classification schemes.

[0004] One is the similarity-based text classification method, which attempts to classify by the similarity between text representation and category label. However, due to the problem that the category label is short and the text to be classified is long, it is often necessary to combine the knowledge base, existing corpus or large language model to expand the category label or the keywords representing the category, and expand the category label into a series of label descriptions or text descriptions. There are also some methods that represent the text to be classified and the label as sparse or dense vector representations in the same space.

[0005] The second is to use other algorithms to process on the basis of unsupervised clustering to achieve the classification effect.

[0006] The third is a transfer-based method that transfers the trained text classification model between different fields or different languages, thereby achieving unsupervised text classification in a specific field or specific language.

[0007] However, similarity-based text classification methods may lead to inaccurate calculation of text similarity due to the problem that the category labels are short and the text to be classified is long. If the category labels are expanded in combination with the knowledge base, existing corpus or large language model, the text classification process will become complicated and time-consuming; if the text to be classified and the label are represented as vector representations in the same space, the semantic information of the text may be lost, affecting the classification effect; in addition, for cases where the text length difference is large, the text classification model may find it difficult to accurately calculate the similarity.

[0008] Methods based on unsupervised clustering often require pre-setting the number of clusters (such as the K value in the K-means algorithm), which may have a significant impact on the final clustering effect. Some classification algorithms do not require pre-setting the number of clusters, but require a lot of computing time. In addition, clustering is usually sensitive to outliers and noise data, which may affect the quality of clustering, and the selection of initial cluster centers has a significant impact on the results, while good initial points are often difficult to determine. When combining classification algorithms for text classification, binary classification algorithms are usually simpler, but the model processing process is complicated when there are a large number of categories.

[0009] For transfer-based methods, the differences between the source domain (or source language) and the target domain (or target language) may lead to poor transfer learning effects; the selection of weights and the measurement of similarity rely on experience, which may require a lot of debugging; the data distribution of the source domain (or source language) and the target domain (or target language) is often different, which may lead to a decrease in the generalization ability of the model in the new domain; negative transfer may occur, that is, the knowledge learned in the source domain (or source language) has a negative impact on the learning of the target domain (or target language).

[0010] Therefore, the classification efficiency and accuracy of existing unsupervised text classification methods are not high. Summary of the invention

[0011] The present invention provides an unsupervised text classification method, device, equipment, storage medium and product, which are used to solve the defects of low classification efficiency and accuracy of the unsupervised text classification method in the prior art.

[0012] The present invention provides an unsupervised text classification method, comprising: obtaining a text set to be classified; dividing the text set to be classified into a sample set to be labeled and a text set to be labeled; classifying and labeling the sample set to be labeled based on a large language model to obtain a labeled sample set; training an initial text classifier based on the labeled sample set to obtain a text classifier; and classifying and labeling the text set to be labeled based on the text classifier to obtain a text classification result.

[0013] According to an unsupervised text classification method provided by the present invention, the large language model includes a first large language model and a second large language model, and the sample set to be labeled includes the text to be labeled; based on the large language model, the sample set to be labeled is classified and labeled to obtain the labeled sample set, including: inputting the text to be labeled in the sample set to the first large language model to obtain a first labeling result of the text to be labeled output by the first large language model; inputting the text to be labeled in the sample set to the second large language model to obtain a second labeling result of the text to be labeled output by the second large language model; judging whether the first labeling result and the second labeling result are consistent; if the first labeling result and the second labeling result are consistent, taking the first labeling result as the final labeling result of the text to be labeled to obtain the labeled sample set.

[0014] According to an unsupervised text classification method provided by the present invention, the large language model includes a first large language model, a second large language model and a third large language model, and the sample set to be labeled includes text to be labeled; based on the large language model, the sample set to be labeled is classified and labeled to obtain the labeled sample set, including: inputting the text to be labeled in the sample set to be labeled into the first large language model to obtain a first labeling result of the text to be labeled output by the first large language model; inputting the text to be labeled in the sample set to be labeled into the second large language model to obtain a second labeling result of the text to be labeled output by the second large language model; judging whether the first labeling result and the second labeling result are consistent; if the first labeling result and the second labeling result are inconsistent, inputting the text to be labeled in the sample set to be labeled into the third large language model to obtain a third labeling result of the text to be labeled output by the third large language model, and taking the third labeling result as the final labeling result of the text to be labeled to obtain the labeled sample set; wherein the model parameter amount of the third large language model is greater than the model parameter amount of the first large language model, and the model parameter amount of the third large language model is greater than the model parameter amount of the second large language model.

[0015] According to an unsupervised text classification method provided by the present invention, a text set to be classified is obtained, including: obtaining an initial text set to be classified; the initial text set to be classified includes an initial text to be annotated; judging whether the length of the initial text to be annotated is greater than or equal to a first preset threshold and less than or equal to a second preset threshold; the second preset threshold is greater than the first preset threshold; if the length of the initial text to be annotated is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, the initial text to be annotated is used as the text to be annotated; if the length of the initial text to be annotated is less than the first preset threshold, data enhancement processing is performed on the initial text to be annotated to obtain the text to be annotated; if the length of the initial text to be annotated is greater than the second preset threshold, a text summary of the initial text to be annotated is generated, and the text summary is used as the text to be annotated; the length of the text summary is greater than or equal to the first preset threshold and less than or equal to the second preset threshold.

[0016] According to an unsupervised text classification method provided by the present invention, the annotated sample set includes a temporary training set and a temporary verification set; based on the annotated sample set, an initial text classifier is trained to obtain a text classifier, including: training the initial text classifier based on the temporary training set, and after the training is completed, verifying the initial text classifier based on the temporary verification set to determine whether the model accuracy of the initial text classifier meets expectations; if the model accuracy of the initial text classifier meets expectations, the text classifier is obtained.

[0017] According to an unsupervised text classification method provided by the present invention, based on a text classifier, a text set to be annotated is classified and annotated, and after obtaining a text classification result, the method further includes: sampling the result of the text set to be annotated to obtain a plurality of texts to be verified; based on a large language model, classifying and annotating the plurality of texts to be verified to obtain a verification classification result of each text to be verified; judging whether the verification classification result of each text to be verified is consistent with the text classification result of each text to be verified; if there is a text to be verified whose verification classification result is inconsistent with the text classification result, storing the text to be verified whose verification classification result is inconsistent with the text classification result into an incremental data set; and optimizing the text classifier based on the incremental data set.

[0018] The present invention also provides a supervised text classification device, comprising: an acquisition module, used to acquire a text set to be classified; a division module, used to divide the text set to be classified into a sample set to be labeled and a text set to be labeled; a labeling module, used to classify and label the sample set to be labeled based on a large language model to obtain a labeled sample set; a training module, used to train an initial text classifier based on the labeled sample set to obtain a text classifier; and a classification module, used to classify and label the text set to be labeled based on the text classifier to obtain a text classification result.

[0019] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned unsupervised text classification methods is implemented.

[0020] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the computer program implements any of the above-mentioned unsupervised text classification methods.

[0021] The present invention also provides a computer program product, comprising a computer program, which implements any of the above-mentioned unsupervised text classification methods when executed by a processor.

[0022] The unsupervised text classification method, device, equipment, storage medium and product provided by the present invention divide the text set to be classified into a sample set to be labeled and a text set to be labeled, first use a large language model to classify and label the sample set to be labeled to obtain a labeled sample set, and then use the labeled sample set to train a text classifier, thereby converting the unsupervised classification into a relatively simple supervised classification, and finally classify and label the text set to be labeled by the trained text classifier to obtain a text classification result, which can avoid the problems of low classification efficiency and classification accuracy caused by complex similarity calculation, difficult clustering setting and poor transfer learning effect, and can effectively improve the text classification efficiency and the accuracy of the text classification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 This is one of the flow charts of the unsupervised text classification method provided by the present invention.

[0025] Figure 2 This is the second flow chart of the unsupervised text classification method provided by the present invention.

[0026] Figure 3 It is a schematic diagram of the process of classification and labeling based on a large language model provided by the present invention.

[0027] Figure 4 It is a flow chart of the intelligent specification of text length provided by the present invention.

[0028] Figure 5 It is a structural schematic diagram of the unsupervised text classification device provided by the present invention.

[0029] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] See also Figure 1, Figure 1 This is one of the flow charts of the unsupervised text classification method provided by the present invention. In this embodiment, the unsupervised text classification method includes steps S110 to S150, and each step is specifically as follows: S110: Obtain a set of texts to be classified.

[0032] The unsupervised text classification method of this embodiment is applicable to the case where the number m of texts to be annotated is much larger than the number n of categories for text classification. Therefore, the set of texts to be classified includes multiple texts to be annotated.

[0033] See also Figure 2 , Figure 2 This is the second flowchart of the unsupervised text classification method provided by the present invention.

[0034] like Figure 2 As shown, considering that the length difference of the text to be annotated will affect the classification result of the model, in order to ensure the text classification quality of the subsequent model, after obtaining the initial text set to be classified (the initial text set to be classified includes multiple initial texts to be annotated), it is necessary to perform intelligent text length standardization on all the initial texts to be annotated in the initial text set to be classified to ensure that all the initial texts to be annotated are close in length.

[0035] Specifically, first set the standardized text length range [Len min , Len max ], among which Len min is the first preset threshold, Len max is the second preset threshold, the second preset threshold Len max Greater than the first preset threshold Len min .

[0036] Furthermore, for each initial text to be annotated in the initial text set to be classified, the length Len of the initial text to be annotated is determined. text Is it greater than or equal to the first preset threshold Len min , and is less than or equal to the second preset threshold Len max .

[0037] If the length of the initial text to be annotated is Len text Greater than or equal to the first preset threshold Len min , and is less than or equal to the second preset threshold Len max , that is, Len min ≤Len text ≤Len max , it means that the length of the initial text to be annotated meets the specification and no processing is required. Therefore, the initial text to be annotated can be directly used as the text to be annotated.

[0038] If the length of the initial text to be annotated is Len text Less than the first preset threshold Len min , that is, Len text <Len min , then the initial text to be annotated can be data enhanced based on the large language model until the length of the initial text to be annotated is Len text Comply with the standardized text length range and obtain the text to be annotated.

[0039] If the length of the initial text to be annotated is Len text Greater than the second preset threshold Len max , that is, Len text >Len max , then a text summary of the initial text to be annotated can be generated based on an automatic summary algorithm or a large language model, and the text summary is used as the text to be annotated; the length of the text summary needs to be greater than or equal to the first preset threshold Len min , and is less than or equal to the second preset threshold Len max .

[0040] Optionally, the automatic summarization algorithm is a Luhn algorithm, a TextRank algorithm, or the like.

[0041] Through intelligent standard processing of text length, the initial texts to be annotated with different text lengths in the initial text set to be classified can be converted into a text set to be classified containing multiple texts to be annotated with similar text lengths.

[0042] S120: Divide the to-be-classified text set into a to-be-annotated sample set and a to-be-annotated text set.

[0043] After obtaining the text set to be classified, it is necessary to group the text set to be classified into a sample set to be labeled and a text set to be labeled.

[0044] Specifically, the unsupervised text classification method of this embodiment is suitable for the case where the number m of texts to be annotated is much larger than the number n of categories of text classification. Therefore, after obtaining the set of texts to be classified, algorithms such as simple random sampling and systematic sampling can be used to select approximately The number of samples to be classified is extracted from the text collection to be classified. The remaining texts to be labeled that are not extracted from the set of texts to be classified constitute the set of texts to be labeled G. t2 .

[0045] Furthermore, The texts to be annotated are evenly divided into two sets to form a temporary training set G t1and temporary check set G v .

[0046] Optionally, α is in the interval [100, 500].

[0047] S130: Based on the large language model, classify and annotate the sample set to be annotated to obtain an annotated sample set.

[0048] Specifically, based on the large language model, each text to be annotated in the annotated sample set is classified and annotated to obtain an annotated sample set.

[0049] S140: Based on the labeled sample set, train the initial text classifier to obtain a text classifier.

[0050] Specifically, based on the labeled sample set, the initial text classifier is trained to obtain a text classifier, thereby converting the unsupervised classification into a relatively simple supervised classification.

[0051] S150: Based on the text classifier, classify and annotate the text set to be annotated to obtain a text classification result.

[0052] Specifically, after obtaining a trained text classifier, the text classifier can be used to classify and annotate each to-be-annotated text in the annotated text set to obtain a text classification result.

[0053] The unsupervised text classification method provided in this embodiment divides the text set to be classified into a sample set to be labeled and a text set to be labeled, first uses a large language model to classify and label the sample set to be labeled to obtain a labeled sample set, and then uses the labeled sample set to train a text classifier, thereby converting the unsupervised classification into a relatively simple supervised classification, and finally uses the trained text classifier to classify and label the text set to be labeled to obtain a text classification result, which can avoid the problems of low classification efficiency and classification accuracy caused by complex similarity calculation, difficult clustering setting, and poor transfer learning effect, and can effectively improve the text classification efficiency and the accuracy of text classification results.

[0054] In some embodiments, the large language model includes a first large language model and a second large language model, and the sample set to be labeled includes text to be labeled; based on the large language model, the sample set to be labeled is classified and labeled to obtain the labeled sample set, including: inputting the text to be labeled in the sample set to be labeled into the first large language model to obtain a first labeling result of the text to be labeled output by the first large language model; inputting the text to be labeled in the sample set to be labeled into the second large language model to obtain a second labeling result of the text to be labeled output by the second large language model; judging whether the first labeling result and the second labeling result are consistent; if the first labeling result and the second labeling result are consistent, taking the first labeling result as the final labeling result of the text to be labeled to obtain the labeled sample set.

[0055] See also Figure 3 , Figure 3 It is a schematic diagram of the process of classification and labeling based on a large language model provided by the present invention.

[0056] like Figure 3 As shown, this embodiment adopts a classification intelligent labeling module including multiple large language models (LLMs), namely the first large language model (LLM1), the second large language model (LLM2) and the third large language model (LLM3), to treat the labeling sample set (i.e., the temporary training set G t1 and temporary check set G v ) are classified and labeled to obtain a labeled sample set.

[0057] Among them, the three large language models can be deployed locally or called through the service interface; the model parameters of the first language model (LLM1) and the second language model (LLM2) are relatively small, for example, not more than 4B, and the first language model (LLM1) and the second language model (LLM2) can be large language models from different manufacturers, so that the two are relatively independent; and the model parameters of the third language model (LLM3) are relatively large, which must be much larger than the model parameters of the first language model (LLM1) and the second language model (LLM2), for example, it can be 72B, 130B, etc. The third language model (LLM3) does not belong to the same manufacturer as the first language model (LLM1) and the second language model (LLM2), and the effect is generally better than the first language model (LLM1) and the second language model (LLM2).

[0058] It should be noted that the more model parameters there are, the greater the amount of model calculation, but the higher the model processing performance, the higher the accuracy of text annotation; the fewer model parameters there are, the smaller the amount of model calculation, but the lower the model processing performance, the lower the accuracy of text annotation.

[0059] like Figure 3As shown in the figure, in order to minimize the overall computational complexity of the three large language models, the first language model (LLM1) and the second language model (LLM2) can be given priority for classification and labeling, and the frequency of use of the third language model (LLM3) can be reduced as much as possible; at the same time, considering that the model performance of the first language model (LLM1) and the second language model (LLM2) is relatively low, the results of text labeling may have large errors. Therefore, the first language model (LLM1) and the second language model (LLM2) can be used simultaneously for classification and labeling, rather than relying solely on the classification and labeling results of a single large language model, to ensure the accuracy of the classification and labeling results, ensure the quality of the training samples, and then ensure the text classification performance of the subsequent text classifier.

[0060] Specifically, for the sample set to be labeled (i.e., the temporary training set G t1 and temporary check set G v ), input the text to be annotated into the first language model, and guide the first language model with the help of the written prompt words to obtain the first annotation result of the text to be annotated output by the first language model. Assuming that the entire category range is recorded as CS, the first annotation result of the text to be annotated output by the first language model can be recorded as C i1 , C i1 It is a specific category in CS.

[0061] Furthermore, the text to be annotated is input into the second largest language model, and the second largest language model is guided by the written prompt words to obtain the second annotation result C of the text to be annotated output by the second largest language model. i2 , C i2 It is a specific category in CS.

[0062] Further, it is determined whether the first labeling result and the second labeling result are consistent, that is, whether C i1 With C i2 Are they consistent?

[0063] If the first annotation result is consistent with the second annotation result, that is, C i1 =C i2 , then the first annotation result C i1 As the final annotation result of the text to be annotated.

[0064] In some embodiments, the large language model includes a first large language model, a second large language model and a third large language model, and the sample set to be labeled includes text to be labeled; based on the large language model, the sample set to be labeled is classified and labeled to obtain the labeled sample set, including: inputting the text to be labeled in the sample set to be labeled into the first large language model to obtain a first labeling result of the text to be labeled output by the first large language model; inputting the text to be labeled in the sample set to be labeled into the second large language model to obtain a second labeling result of the text to be labeled output by the second large language model; judging whether the first labeling result and the second labeling result are consistent; if the first labeling result and the second labeling result are inconsistent, inputting the text to be labeled in the sample set to be labeled into the third large language model to obtain a third labeling result of the text to be labeled output by the third large language model, and using the third labeling result as the final labeling result of the text to be labeled to obtain the labeled sample set; wherein the model parameter amount of the third large language model is greater than the model parameter amount of the first large language model, and the model parameter amount of the third large language model is greater than the model parameter amount of the second large language model.

[0065] Please continue reading Figure 3 , it can be understood that for the same text to be annotated, the first annotation result output by the first language model (LLM1) can be recorded as C i1 And the second annotation result C output by the second largest language model (LLM2) i2 May be inconsistent, that is, C i1 ≠C i2 Since the first language model (LLM1) and the second language model (LLM2) are both small parameter models, the performance of the first language model (LLM1) and the second language model (LLM2) are both low, and the labeling results of both may not be reliable. At this time, in order to ensure the accuracy of the classification labeling results, a large parameter model, that is, the third language model (LLM3), can be used for more accurate labeling.

[0066] Specifically, for the sample set to be labeled (i.e., the temporary training set G t1 and temporary check set G v ) in each text to be annotated, if its first annotation result C i1 and the second annotation result C i2 Inconsistent, i.e. C i1 ≠C i2 , then the text to be annotated is input into the third language model to obtain the third annotation result C of the text to be annotated output by the third language model i3 , and the third annotation result C i3 As the final annotation result of the text to be annotated.

[0067] After obtaining the sample set to be labeled (i.e., the temporary training set G t1and temporary check set G v ), these texts to be annotated with the final annotation results are used as the annotation sample set.

[0068] In some embodiments, obtaining a set of texts to be classified includes: obtaining an initial set of texts to be classified; the initial set of texts to be classified includes an initial text to be annotated; determining whether the length of the initial text to be annotated is greater than or equal to a first preset threshold and less than or equal to a second preset threshold; the second preset threshold is greater than the first preset threshold; if the length of the initial text to be annotated is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, the initial text to be annotated is used as the text to be annotated; if the length of the initial text to be annotated is less than the first preset threshold, data enhancement processing is performed on the initial text to be annotated to obtain the text to be annotated; if the length of the initial text to be annotated is greater than the second preset threshold, a text summary of the initial text to be annotated is generated, and the text summary is used as the text to be annotated; the length of the text summary is greater than or equal to the first preset threshold and less than or equal to the second preset threshold.

[0069] See also Figure 4 , Figure 4 It is a flow chart of the intelligent specification of text length provided by the present invention.

[0070] like Figure 4 As shown, considering that the length difference of the text to be annotated will affect the classification result of the model, in order to ensure the text classification quality of the subsequent model, after obtaining the initial text set to be classified (the initial text set to be classified includes multiple initial texts to be annotated), it is necessary to perform intelligent text length standardization on all the initial texts to be annotated in the initial text set to be classified to ensure that all the initial texts to be annotated are close in length.

[0071] Specifically, first set the standardized text length range [Len min , Len max ], among which Len min is the first preset threshold, Len max is the second preset threshold, the second preset threshold Len max Greater than the first preset threshold Len min .

[0072] Furthermore, for each initial text to be annotated in the initial text set to be classified, the length Len of the initial text to be annotated is determined. text Is it greater than or equal to the first preset threshold Len min , and is less than or equal to the second preset threshold Len max .

[0073] If the length of the initial text to be annotated is Lentext Greater than or equal to the first preset threshold Len min , and is less than or equal to the second preset threshold Len max , that is, Len min ≤Len text ≤Len max , it means that the length of the initial text to be annotated meets the specification and no processing is required. Therefore, the initial text to be annotated can be directly used as the text to be annotated.

[0074] If the length of the initial text to be annotated is Len text Less than the first preset threshold Len min , that is, Len text <Len min , then the initial text to be annotated can be data enhanced based on the large language model until the length of the initial text to be annotated is Len text Comply with the standardized text length range and obtain the text to be annotated.

[0075] If the length of the initial text to be annotated is Len text Greater than the second preset threshold Len max , that is, Len text >Len max , then a text summary of the initial text to be annotated can be generated based on an automatic summary algorithm or a large language model, and the text summary is used as the text to be annotated; the length of the text summary needs to be greater than or equal to the first preset threshold Len min , and is less than or equal to the second preset threshold Len max .

[0076] Optionally, the automatic summarization algorithm is a Luhn algorithm, a TextRank algorithm, or the like.

[0077] Through intelligent text length standardization processing, the initial texts to be annotated with different text lengths in the initial text set to be classified can be converted into a text set to be classified containing multiple texts to be annotated with similar text lengths.

[0078] In some embodiments, the annotated sample set includes a temporary training set and a temporary verification set; based on the annotated sample set, an initial text classifier is trained to obtain a text classifier, including: training the initial text classifier based on the temporary training set, and after the training is completed, verifying the initial text classifier based on the temporary verification set to determine whether the model accuracy of the initial text classifier meets expectations; if the model accuracy of the initial text classifier meets expectations, the text classifier is obtained.

[0079] After obtaining the temporary training set G t1 and temporary check set G v Afterwards, we can use the labeled temporary training set Gt1 Train the initial text classifier and after the training is completed, use the temporary verification set G v Verify the initial text classifier to determine whether the model accuracy of the initial text classifier meets expectations.

[0080] If the model accuracy of the initial text classifier meets expectations, a trained text classifier is obtained. The trained text classifier is a model that can automatically classify unlabeled texts within the entire category range CS.

[0081] Optionally, the initial text classifier is a model built based on a machine learning algorithm such as SVM, KNN, etc. / a deep learning algorithm such as RNN, LSTM, etc., or it can be a model such as BERT, ERNIE, etc. specifically used for natural language processing.

[0082] In some embodiments, based on a text classifier, the text set to be annotated is classified and annotated to obtain the text classification result, and the method further includes: sampling the results of the text set to be annotated to obtain multiple texts to be verified; based on a large language model, classifying and annotating the multiple texts to be verified to obtain a verification classification result of each text to be verified; judging whether the verification classification result of each text to be verified is consistent with the text classification result of each text to be verified; if there is a text to be verified whose verification classification result is inconsistent with the text classification result, storing the text to be verified whose verification classification result is inconsistent with the text classification result into an incremental data set; and optimizing the text classifier based on the incremental data set.

[0083] After obtaining the trained text classifier, use the text classifier to treat the annotated text set G t2 Classify and annotate each text to be annotated in the text set G t2 The text classification results of each text to be annotated in .

[0084] Understandably, inaccurate classification may still occur during the application stage of the text classifier, especially in the initial stage of application. Therefore, after the text classifier completes the classification labeling, the text classifier needs to be continuously optimized.

[0085] Specifically, in the text classification process, a specific strategy can be adopted. t2 After a total of j texts to be annotated are collected, the annotated text set G can be t2 Perform result sampling and select the last k texts from the total number of j texts to be annotated as the texts to be verified.

[0086] Furthermore, a classification intelligent annotation module including multiple large language models (LLMs) is used to reclassify and annotate the k texts to be verified, and the verification classification results of each text to be verified are obtained.

[0087] Further, it is determined whether the verification classification result of each text to be verified is consistent with the text classification result of each text to be verified.

[0088] If there is a text to be verified whose verification classification result is inconsistent with the text classification result, the text to be verified whose verification classification result is inconsistent with the text classification result is stored in the incremental data set G s .

[0089] Further, calculate the incremental data set G s The calculation formula of the incremental coefficient β is as follows: β = G s / (G t1 +G v ).

[0090] If the increment coefficient β is greater than or equal to the third preset threshold, the incremental data set G s The texts to be verified in the set are evenly distributed to the temporary training set G t1 and temporary check set G v And use the updated temporary training set G t1 and temporary check set G v , perform incremental training on the text classifier. If the algorithm used by the text classifier does not support incremental training, it can be retrained to obtain an optimized text classifier. Subsequent text classification annotation will be performed using the optimized text classifier.

[0091] Optionally, the third preset threshold is 0.1.

[0092] Optionally, the classification results of the classification intelligent annotation modules at different stages and the results of the text classifier are summarized and output as the final classification result of the entire set of texts to be classified.

[0093] The unsupervised text classification method provided by the present invention reasonably schedules and utilizes large language models, pre-trained models and traditional machine learning algorithms, and reduces calculations as much as possible while ensuring the text classification effect, thereby saving time and cost. Taking 100,000 text data and 9 categories of classification problems as an example, on the same server, under the premise of consistent intelligent standard time for text length, only considering the classification time, compared with about 117 hours to 373 hours required for only using a single or multiple large models, the unsupervised text classification method of the present invention only takes about 8 hours and 25 minutes, and under the same number of classification categories, the more texts to be annotated, the shorter the time required for a single text classification; in addition, in the process of intelligent annotation, the present invention uses three large language models with parameter sizes of 6B, 7B, and 13B respectively, which only requires 1 32GV100GPU, which can save computing resources.

[0094] The present invention also provides an unsupervised text classification device. Figure 5 , Figure 5 5 is a schematic diagram of the structure of the unsupervised text classification device provided by the present invention. In this embodiment, the unsupervised text classification device includes an acquisition module 510 , a division module 520 , a labeling module 530 , a training module 540 and a classification module 550 .

[0095] The acquisition module 510 is used to acquire a set of texts to be classified.

[0096] The division module 520 is used to divide the to-be-classified text set into a to-be-annotated sample set and a to-be-annotated text set.

[0097] The labeling module 530 is used to classify and label the sample set to be labeled based on the large language model to obtain a labeled sample set.

[0098] The training module 540 is used to train the initial text classifier based on the labeled sample set to obtain a text classifier.

[0099] The classification module 550 is used to classify and annotate the text set to be annotated based on a text classifier to obtain a text classification result.

[0100] In some embodiments, the large language model includes a first large language model and a second large language model, and the sample set to be labeled includes text to be labeled.

[0101] The annotation module 530 is used to input the text to be annotated in the sample set to be annotated into the first largest language model to obtain a first annotation result of the text to be annotated output by the first largest language model; input the text to be annotated in the sample set to be annotated into the second largest language model to obtain a second annotation result of the text to be annotated output by the second largest language model; determine whether the first annotation result and the second annotation result are consistent; if the first annotation result and the second annotation result are consistent, use the first annotation result as the final annotation result of the text to be annotated to obtain an annotation sample set.

[0102] In some embodiments, the large language model includes a first large language model, a second large language model, and a third large language model, and the sample set to be labeled includes text to be labeled.

[0103] The annotation module 530 is used to input the text to be annotated in the sample set to be annotated into the first largest language model to obtain a first annotation result of the text to be annotated output by the first largest language model; input the text to be annotated in the sample set to be annotated into the second largest language model to obtain a second annotation result of the text to be annotated output by the second largest language model; determine whether the first annotation result is consistent with the second annotation result; if the first annotation result and the second annotation result are inconsistent, input the text to be annotated in the sample set to be annotated into the third largest language model to obtain a third annotation result of the text to be annotated output by the third largest language model, and use the third annotation result as the final annotation result of the text to be annotated to obtain an annotation sample set; wherein the model parameter amount of the third largest language model is greater than the model parameter amount of the first largest language model, and the model parameter amount of the third largest language model is greater than the model parameter amount of the second largest language model.

[0104] In some embodiments, the acquisition module 510 is used to obtain an initial set of texts to be classified; the initial set of texts to be classified includes an initial text to be annotated; it is determined whether the length of the initial text to be annotated is greater than or equal to a first preset threshold and less than or equal to a second preset threshold; the second preset threshold is greater than the first preset threshold; if the length of the initial text to be annotated is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, the initial text to be annotated is used as the text to be annotated; if the length of the initial text to be annotated is less than the first preset threshold, data enhancement processing is performed on the initial text to be annotated to obtain the text to be annotated; if the length of the initial text to be annotated is greater than the second preset threshold, a text summary of the initial text to be annotated is generated, and the text summary is used as the text to be annotated; the length of the text summary is greater than or equal to the first preset threshold and less than or equal to the second preset threshold.

[0105] In some embodiments, the labeled sample set includes a temporary training set and a temporary verification set.

[0106] The training module 540 is used to train the initial text classifier based on the temporary training set, and after the training is completed, verify the initial text classifier based on the temporary verification set to determine whether the model accuracy of the initial text classifier meets expectations; if the model accuracy of the initial text classifier meets expectations, the text classifier is obtained.

[0107] In some embodiments, the training module 540 is used to sample the results of the text set to be annotated to obtain multiple texts to be verified; based on the large language model, classify and annotate the multiple texts to be verified to obtain the verification classification result of each text to be verified; determine whether the verification classification result of each text to be verified is consistent with the text classification result of each text to be verified; if there are texts to be verified whose verification classification results are inconsistent with the text classification results, then store the texts to be verified whose verification classification results are inconsistent with the text classification results into an incremental data set; based on the incremental data set, optimize the text classifier.

[0108] The invention also provides an electronic device. Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the unsupervised text classification method.

[0109] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0110] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the unsupervised text classification method provided by the above methods is implemented.

[0111] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the unsupervised text classification method provided by the above methods.

[0112] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An unsupervised text classification method, characterized in that: include: Get the text collection to be classified; Dividing the to-be-classified text set into a to-be-annotated sample set and a to-be-annotated text set; Based on the large language model, classify and annotate the sample set to be annotated to obtain an annotated sample set; Based on the labeled sample set, an initial text classifier is trained to obtain a text classifier; Based on the text classifier, the text set to be annotated is classified and annotated to obtain a text classification result.

2. The unsupervised text classification method according to claim 1, characterized in that: The large language model includes a first large language model and a second large language model, and the sample set to be labeled includes text to be labeled; The method of classifying and labeling the sample set to be labeled based on the large language model to obtain the labeled sample set includes: Inputting the text to be annotated in the sample set to be annotated into the first large language model, and obtaining a first annotation result of the text to be annotated output by the first large language model; Inputting the text to be annotated in the sample set to be annotated into the second largest language model, and obtaining a second annotation result of the text to be annotated output by the second largest language model; Determining whether the first labeling result and the second labeling result are consistent; If the first annotation result is consistent with the second annotation result, the first annotation result is used as the final annotation result of the text to be annotated to obtain an annotation sample set.

3. The unsupervised text classification method according to claim 1, characterized in that: The large language model includes a first large language model, a second large language model and a third large language model, and the sample set to be labeled includes text to be labeled; The method of classifying and labeling the sample set to be labeled based on the large language model to obtain the labeled sample set includes: Inputting the text to be annotated in the sample set to be annotated into the first large language model, and obtaining a first annotation result of the text to be annotated output by the first large language model; Inputting the text to be annotated in the sample set to be annotated into the second largest language model, and obtaining a second annotation result of the text to be annotated output by the second largest language model; Determining whether the first labeling result and the second labeling result are consistent; If the first annotation result and the second annotation result are inconsistent, inputting the text to be annotated in the sample set to be annotated into the third language model, obtaining a third annotation result of the text to be annotated output by the third language model, and using the third annotation result as the final annotation result of the text to be annotated to obtain an annotation sample set; Among them, the model parameter amount of the third largest language model is greater than the model parameter amount of the first largest language model, and the model parameter amount of the third largest language model is greater than the model parameter amount of the second largest language model.

4. The unsupervised text classification method according to claim 1, characterized in that: The step of obtaining a set of texts to be classified includes: Acquire an initial set of texts to be classified; the initial set of texts to be classified includes initial texts to be annotated; Determine whether the length of the initial text to be annotated is greater than or equal to a first preset threshold and less than or equal to a second preset threshold; the second preset threshold is greater than the first preset threshold; If the length of the initial text to be annotated is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, the initial text to be annotated is used as the text to be annotated; If the length of the initial text to be annotated is less than a first preset threshold, performing data enhancement processing on the initial text to be annotated to obtain the text to be annotated; If the length of the initial text to be annotated is greater than a second preset threshold, a text summary of the initial text to be annotated is generated, and the text summary is used as the text to be annotated; the length of the text summary is greater than or equal to the first preset threshold and less than or equal to the second preset threshold.

5. The unsupervised text classification method according to claim 1, characterized in that: The labeled sample set includes a temporary training set and a temporary verification set; The step of training an initial text classifier based on the labeled sample set to obtain a text classifier includes: Training the initial text classifier based on the temporary training set, and after the training is completed, verifying the initial text classifier based on the temporary verification set to determine whether the model accuracy of the initial text classifier meets expectations; If the model accuracy of the initial text classifier meets expectations, the text classifier is obtained.

6. The unsupervised text classification method according to claim 5, characterized in that: After classifying and labeling the to-be-labeled text set based on the text classifier and obtaining the text classification result, the method further includes: Sampling the results of the text set to be annotated to obtain multiple texts to be verified; Based on the large language model, classify and annotate the plurality of texts to be verified to obtain a verification classification result of each text to be verified; Determining whether the verification classification result of each of the texts to be verified is consistent with the text classification result of each of the texts to be verified; If there is a text to be verified whose verification classification result is inconsistent with the text classification result, storing the text to be verified whose verification classification result is inconsistent with the text classification result into the incremental data set; The text classifier is optimized based on the incremental data set.

7. An unsupervised text classification device, characterized in that: include: An acquisition module is used to obtain a set of texts to be classified; A division module, used for dividing the to-be-classified text set into a to-be-annotated sample set and a to-be-annotated text set; A labeling module, used for classifying and labeling the sample set to be labeled based on a large language model to obtain a labeled sample set; A training module, used to train an initial text classifier based on the annotated sample set to obtain a text classifier; The classification module is used to classify and annotate the text set to be annotated based on the text classifier to obtain a text classification result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the unsupervised text classification method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the unsupervised text classification method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the unsupervised text classification method according to any one of claims 1 to 6 is implemented.

Images