A text classification method and related apparatus

Abstract

The application discloses a text classification method and related device, which can be applied to an artificial intelligence scene. A first classification model and a second classification model are subjected to model training based on knowledge distillation by using a plurality of first training texts and category label data labeled for each first training text to obtain a preset classification model. Each text to be classified in the plurality of texts to be classified is input into the preset classification model for category prediction, and a first distribution probability of a plurality of preset categories to which each text to be classified belongs is output. The plurality of texts to be classified are clustered to obtain a second distribution probability of the plurality of preset categories to which each text to be classified belongs. The first distribution probability and the second distribution probability are fused for each text to be classified to obtain a fusion distribution probability of the plurality of preset categories to which the text to be classified belongs. A preset category corresponding to a maximum probability component in the fusion distribution probability is taken as a classification category to which the text to be classified belongs, the classification accuracy is improved, and the classification effect is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a text classification method and related apparatus. Background Technology

[0002] With the rapid development of text processing technology, in some practical business application scenarios, it is necessary to first classify the text to be processed. The text to be processed is then classified based on the classification category, which makes it easier to process the text.

[0003] The relevant technology can obtain a text classification model by pre-training a deep learning model with training text containing category label data, inputting the text to be classified into the text classification model to predict the category, and obtaining the predicted category to which the text to be classified belongs as the classification category to which the text to be classified belongs.

[0004] However, when the complexity of the deep learning model is high, the deployment requirements of the text classification model are high, and when the complexity of the deep learning model is low, the classification accuracy of the text classification model is low. Moreover, a small total number of training texts or an imbalance of categories can easily lead to a weak classification ability of the text classification model. In other words, the classification effect of the text to be classified in the above methods is poor. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a text classification method and related apparatus, which can improve the classification accuracy of the text to be classified, thereby enhancing the classification effect of the text to be classified.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] On the one hand, embodiments of this application provide a text classification method, the method comprising:

[0008] The category of each text to be classified is predicted by a preset classification model to obtain the first distribution probability of each text to be classified belonging to multiple preset categories. The preset classification model is obtained by training the first classification model and the second classification model based on knowledge distillation based on multiple first training texts and the category label data of each first training text. The model complexity of the first classification model is greater than that of the second classification model.

[0009] Cluster the multiple texts to be classified to obtain the second distribution probability of each text to be classified belonging to the multiple preset categories;

[0010] For each text to be classified, the first distribution probability of the text to be classified belonging to the plurality of preset categories and the second distribution probability of the text to be classified belonging to the plurality of preset categories are fused to obtain the fused distribution probability of the text to be classified belonging to the plurality of preset categories;

[0011] The classification category of the text to be classified is determined based on the preset category corresponding to the highest probability component in the fusion distribution probability.

[0012] Optionally, the step of fusing the first distribution probability of the text to be classified to the plurality of preset categories and the second distribution probability of the text to be classified to the plurality of preset categories to obtain the fused distribution probability of the text to be classified to the plurality of preset categories includes:

[0013] Determine the first weight corresponding to the first probability distribution and the second weight corresponding to the second probability distribution; the sum of the first weight and the second weight is 1;

[0014] The first distribution probability, the first weight, the second distribution probability, and the second weight of the multiple preset categories to which the text to be classified belong are weighted to obtain the fused distribution probability of the multiple preset categories to which the text to be classified belongs.

[0015] Optionally, the training steps of the preset classification model include:

[0016] For each first training text, the first pre-trained model in the first classification model is used to predict the category of the first training text, and the third distribution probability of the first training text belonging to the multiple preset categories is obtained.

[0017] By using the normalization layer in the first classification model, the third distribution probability is normalized based on knowledge distillation to obtain the target distribution probability of the multiple preset categories to which the first training text belongs.

[0018] The second pre-trained model in the second classification model is used to predict the category of each first training text to obtain the fourth distribution probability of the multiple preset categories to which the first training text belongs; the model complexity of the first pre-trained model is greater than that of the second pre-trained model.

[0019] By using the first normalization layer in the second classification model, the fourth distribution probability is normalized based on knowledge distillation to obtain the first predicted probability of the first training text belonging to the multiple preset categories.

[0020] The fourth distribution probability is normalized by the second normalization layer in the second classification model to obtain the second predicted probability of the first training text belonging to the multiple preset categories.

[0021] Based on the first predicted probability, the target distribution probability, the second predicted probability, the category label data of the first training text annotation, and the preset loss function of the second classification model, train the model parameters of the second classification model until the model converges;

[0022] The second pre-trained model and the second normalization layer in the trained second classification model are determined as the preset classification model.

[0023] Optionally, the preset loss function includes a first loss function and a second loss function; training the model parameters of the second classification model until the model converges based on the first predicted probability, the target distribution probability, the second predicted probability, the category label data of the first training text annotation, and the preset loss function of the second classification model includes:

[0024] Based on the first predicted probability, the target distribution probability, and the first loss function, determine the first loss corresponding to the first training text;

[0025] Based on the second predicted probability, the category label data, and the second loss function, determine the second loss corresponding to the first training text;

[0026] The first loss, the third weight corresponding to the first loss, the second loss, and the fourth weight corresponding to the second loss are weighted to obtain the total loss corresponding to the first training text; the sum of the third weight and the fourth weight is 1.

[0027] The model parameters of the second classification model are trained based on the total loss until the model converges.

[0028] Optionally, the steps for obtaining the plurality of first training texts specifically include:

[0029] Randomly sample multiple historical texts to obtain multiple first training texts; the total number of the multiple first training texts is greater than or equal to the first number.

[0030] Optionally, the step of obtaining the category label data for each first training text specifically includes:

[0031] In response to the first object performing a category labeling operation on each of the multiple first training texts according to the preset classification business requirements, the category label data of each of the first training texts is obtained.

[0032] Optionally, before training the preset classification model, the method further includes:

[0033] Randomly sample the plurality of first training texts to obtain a plurality of texts to be annotated; the total number of the plurality of texts to be annotated is greater than or equal to a second number, and the second number is less than the first number.

[0034] In response to the second object's category labeling operation for each text to be annotated according to the preset classification business requirements, the annotation category data of each text to be annotated is obtained;

[0035] If the consistency between the multiple category label data corresponding to the multiple texts to be annotated and the multiple annotation category data corresponding to the multiple texts to be annotated is greater than or equal to a preset consistency level, it is determined that the category label data of each first training text is correctly labeled.

[0036] If the consistency level is less than the preset consistency level, update the category label data of each first training text.

[0037] Optionally, before predicting the category of each text to be classified among multiple texts using a preset classification model, the method further includes:

[0038] The preset classification model is used to predict the category of each test text among a plurality of test texts that are different from the plurality of first training texts, and a third prediction probability of the plurality of preset categories to which each test text belongs is obtained.

[0039] Based on the third predicted probability of each test text belonging to multiple preset categories, the category label data of each test text, and preset verification indicators, the preset classification model is validated to obtain the model validation result of the preset classification model.

[0040] If the model verification result does not meet the preset verification conditions, the multiple first training texts are updated according to the multiple historical texts to obtain multiple second training texts;

[0041] Based on the plurality of second training texts and the category label data annotated for each second training text, the first classification model and the second classification model are trained using knowledge distillation, and the preset classification model is updated.

[0042] Optionally, the preset verification metrics include any one of confusion matrix, precision, and recall.

[0043] Optionally, the step of clustering the plurality of texts to be classified to obtain a second distribution probability of each text to be classified belonging to the plurality of preset categories includes:

[0044] Determine multiple initial center points corresponding to the multiple preset categories;

[0045] The multiple texts to be classified are clustered based on the distance between each text to be classified and each initial centroid to determine the preset category to which each text to be classified belongs;

[0046] Each initial centroid is updated based on multiple unclassified texts clustered under a preset category corresponding to each initial centroid.

[0047] Repeat the process of clustering the multiple texts to be classified based on the distance between each text to be classified and each initial centroid until the clustering converges.

[0048] Based on the preset category to which each text to be classified belongs after clustering convergence, determine the second distribution probability of the multiple preset categories to which each text to be classified belongs.

[0049] Optionally, when the text classification method is applied to a text classification response scenario, the method further includes:

[0050] Based on the category to which the text to be classified belongs, determine the text to be replied to that matches the category.

[0051] On the other hand, embodiments of this application provide a text classification device, the device comprising: a prediction unit, a clustering unit, a fusion unit, and a determination unit;

[0052] The prediction unit is used to predict the category of each text to be classified among multiple texts to be classified by using a preset classification model, and to obtain the first distribution probability of multiple preset categories to which each text to be classified belongs; the preset classification model is obtained by training a first classification model and a second classification model based on knowledge distillation based on multiple first training texts and the category label data labeled in each first training text, and the model complexity of the first classification model is greater than that of the second classification model.

[0053] The clustering unit is used to perform clustering processing on the plurality of texts to be classified, and to obtain the second distribution probability of the plurality of preset categories to which each text to be classified belongs;

[0054] The fusion unit is used to perform fusion processing on the first distribution probability of the multiple preset categories to which the text to be classified belongs and the second distribution probability of the multiple preset categories to which the text to be classified belongs for each text to be classified, so as to obtain the fused distribution probability of the multiple preset categories to which the text to be classified belongs.

[0055] The determining unit is used to determine the classification category to which the text to be classified belongs based on the preset category corresponding to the maximum probability component in the fusion distribution probability.

[0056] Optional, fusion unit, specifically used for:

[0057] Determine the first weight corresponding to the first probability distribution and the second weight corresponding to the second probability distribution; the sum of the first weight and the second weight is 1;

[0058] The first distribution probability and first weight of the multiple preset categories to which the text to be classified belongs, the second distribution probability and second weight of the multiple preset categories to which the text to be classified belongs, are weighted to obtain the fused distribution probability of the multiple preset categories to which the text to be classified belongs.

[0059] Optionally, the device may also include: a training unit;

[0060] The training unit is specifically used to predict the category of each first training text by using the first pre-trained model in the first classification model, and to obtain the third distribution probability of the first training text belonging to multiple preset categories.

[0061] By using the normalization layer in the first classification model, the third distribution probability is normalized based on knowledge distillation to obtain the target distribution probability of the first training text belonging to multiple preset categories.

[0062] The second pre-trained model in the second classification model is used to predict the category of each first training text, and obtain the fourth distribution probability of the first training text belonging to multiple preset categories; the model complexity of the first pre-trained model is greater than that of the second pre-trained model.

[0063] By using the first normalization layer in the second classification model, the fourth distribution probability is normalized based on knowledge distillation to obtain the first predicted probability of the first training text belonging to multiple preset categories.

[0064] The fourth distribution probability is normalized by the second normalization layer in the second classification model to obtain the second predicted probability of the first training text belonging to multiple preset categories.

[0065] Based on the first predicted probability, the target distribution probability, the second predicted probability, the category label data of the first training text annotation, and the preset loss function of the second classification model, train the model parameters of the second classification model until the model converges.

[0066] The second pre-trained model and the second normalization layer in the trained second classification model are determined as the preset classification model.

[0067] Optional, training units, specifically used for:

[0068] Based on the first prediction probability, the target distribution probability, and the first loss function, determine the first loss corresponding to the first training text;

[0069] Based on the second predicted probability, the category label data, and the second loss function, determine the second loss corresponding to the first training text;

[0070] The first loss, the third weight corresponding to the first loss, the second loss, and the fourth weight corresponding to the second loss are weighted to obtain the total loss corresponding to the first training text; the sum of the third weight and the fourth weight is 1.

[0071] Train the parameters of the second classification model based on the total loss until the model converges.

[0072] Optionally, the device may also include: a sampling unit;

[0073] The sampling unit is used to randomly sample multiple historical texts collected to obtain multiple first training texts; the total number of multiple first training texts is greater than or equal to the first number.

[0074] Optionally, the device may also include: a labeling unit;

[0075] The annotation unit is used to respond to the first object's category annotation operation for each first training text according to the preset classification business requirements corresponding to multiple first training texts, and to obtain the category label data of each first training text.

[0076] Optionally, the device may further include: a first update unit;

[0077] The sampling unit is also used to randomly sample multiple first training texts to obtain multiple texts to be annotated; the total number of multiple texts to be annotated is greater than or equal to the second number, and the second number is less than the first number;

[0078] The annotation unit is also used to respond to the second object's category annotation operation on each text to be annotated according to the preset classification business requirements, and to obtain the annotation category data of each text to be annotated;

[0079] The determining unit is also used to determine that the category label data of each first training text is correctly labeled if the consistency between the multiple category label data corresponding to multiple texts to be annotated and the multiple annotation category data corresponding to multiple texts to be annotated is greater than or equal to a preset consistency level.

[0080] The first update unit is used to update the category label data of each first training text if the consistency level is less than the preset consistency level.

[0081] Optionally, the device may further include: a verification unit and a second update unit;

[0082] The prediction unit is also used to predict the category of each test text among multiple test texts that are different from multiple first training texts by using a preset classification model, and to obtain the third prediction probability of each test text belonging to multiple preset categories.

[0083] The verification unit is used to perform model verification on the preset classification model based on the third prediction probability of each test text to which it belongs, the category label data of each test text, and the preset verification index, and to obtain the model verification result of the preset classification model.

[0084] The second update unit is used to update multiple first training texts based on multiple historical texts to obtain multiple second training texts if the model validation results do not meet the preset validation conditions.

[0085] The second update unit is also used to perform knowledge distillation-based model training on the first classification model and the second classification model based on multiple second training texts and the category label data annotated for each second training text, and update the preset classification model.

[0086] Optionally, preset validation metrics include any one of confusion matrix, precision, and recall.

[0087] Optional, clustering units, specifically used for:

[0088] Determine multiple initial center points corresponding to multiple preset categories;

[0089] Clustering is performed on multiple texts to be classified based on the distance between each text to be classified and each initial centroid to determine the preset category to which each text to be classified belongs;

[0090] Update each initial centroid based on multiple unclassified texts clustered under the preset category corresponding to each initial centroid;

[0091] Repeat the clustering process based on the distance between each text to be classified and each initial centroid until the clustering converges;

[0092] Based on the preset category to which each text to be classified belongs after clustering convergence, determine the second distribution probability of each text to be classified belonging to multiple preset categories.

[0093] Optionally, when the text classification method is applied to a text classification response scenario, the unit is determined, and the corresponding text to be replied to is determined based on the classification category to which the text to be classified belongs.

[0094] On the other hand, embodiments of this application provide a computer device, the computer device including a processor and a memory:

[0095] The memory is used to store computer programs and to transfer the computer programs to the processor;

[0096] The processor is configured to execute the method described in any of the foregoing aspects according to instructions in the computer program.

[0097] On the other hand, embodiments of this application provide a computer-readable storage medium for storing a computer program for performing the methods described in any of the foregoing aspects.

[0098] On the other hand, embodiments of this application provide a computer program product, including a computer program that, when run on a computer device, causes the computer device to execute the method described in any of the foregoing aspects.

[0099] As can be seen from the above technical solution, firstly, a pre-defined classification model is obtained by pre-training the first and second classification models using knowledge distillation based on multiple first training texts and the category label data annotated for each first training text. The first classification model has a higher model complexity than the second classification model. Then, each text to be classified is input into the pre-defined classification model for category prediction, outputting the first distribution probability of each text belonging to multiple pre-defined categories. Since the knowledge distillation-based model training method reduces model complexity while minimizing the reduction in classification accuracy, the pre-defined classification model can, to a certain extent, guarantee the classification performance of the texts to be classified. Furthermore, clustering multiple texts to be classified yields the second distribution probability of each text belonging to multiple pre-defined categories. This clustering method can compensate for the weakness in classification ability of the pre-defined classification model caused by a small total number of first training texts or an imbalance in categories. Then, for each text to be classified, the first distribution probability and the second distribution probability of the text belonging to multiple preset categories are fused to obtain the fused distribution probability of the text belonging to multiple preset categories. This fusion method makes the fused distribution probability more accurate than the first and second distribution probabilities. Finally, the preset category corresponding to the highest probability component in the fused distribution probability is taken as the classification category of the text to be classified, which can improve the classification accuracy of the text and thus improve the classification effect. Attached Figure Description

[0100] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0101] Figure 1 A schematic diagram of the system architecture of a text classification method provided in this application embodiment;

[0102] Figure 2 A flowchart illustrating a text classification method provided in this application embodiment;

[0103] Figure 3 A schematic diagram of LSTM-based text classification provided for an embodiment of this application;

[0104] Figure 4 A text classification diagram based on textCNN is provided for an embodiment of this application;

[0105] Figure 5 A schematic diagram of BERT-based text classification provided for embodiments of this application;

[0106] Figure 6 This is a schematic diagram illustrating knowledge distillation-based model training for a first classification model and a second classification model, provided as an embodiment of this application.

[0107] Figure 7 A schematic diagram of the model architecture of a preset classification model provided in this application embodiment;

[0108] Figure 8 A schematic diagram of a confusion matrix provided in an embodiment of this application;

[0109] Figure 9 A flowchart illustrating a text classification method tailored to a real-world application scenario, as provided in this application embodiment;

[0110] Figure 10 A structural diagram of a text classification device provided in an embodiment of this application;

[0111] Figure 11 A structural diagram of a server provided in an embodiment of this application;

[0112] Figure 12 This is a structural diagram of a terminal provided in an embodiment of this application. Detailed Implementation

[0113] The embodiments of this application will now be described with reference to the accompanying drawings.

[0114] At present, text classification of text to be classified means: pre-training a deep learning model with training text containing category labels to obtain a text classification model, inputting the text to be classified into the text classification model to predict the category, and obtaining the predicted category to which the text to be classified belongs as the classification category to which the text to be classified belongs.

[0115] However, research has found that when the complexity of the deep learning model is high, the deployment requirements of the text classification model are high, and when the complexity of the deep learning model is low, the classification accuracy of the text classification model is low. In addition, a small total number of training texts or an imbalance of categories can easily lead to a weak classification ability of the text classification model. In other words, the classification effect of the text to be classified in the above methods is poor.

[0116] This application provides a text classification method and related apparatus. A pre-trained classification model is obtained by using multiple first training texts and category label data labeled for each first training text, and then training a first classification model and a second classification model using knowledge distillation. The first classification model has a higher model complexity than the second classification model. Each text to be classified is input into the pre-trained classification model for category prediction, outputting a first distribution probability of each text belonging to multiple pre-trained categories. Because the knowledge distillation-based model training method reduces model complexity without compromising classification accuracy, the pre-trained classification model can, to a certain extent, guarantee the classification performance of the texts to be classified. Furthermore, clustering the multiple texts to be classified yields a second distribution probability of each text belonging to multiple pre-trained categories. This clustering method can compensate for the weak classification ability of the pre-trained classification model caused by a small total number of first training texts or category imbalance. For each text to be classified, the first probability distribution and the second probability distribution of the text to be classified belonging to multiple preset categories are fused to obtain the fused probability distribution of the text to be classified belonging to multiple preset categories. This fusion method makes the fused probability distribution more accurate than the first and second probability distributions. Taking the preset category corresponding to the highest probability component in the fused probability distribution as the classification category of the text to be classified can improve the classification accuracy of the text to be classified, thereby improving the classification effect.

[0117] Next, the system architecture of the text classification method will be introduced. See [link / reference] Figure 1 , Figure 1 This is a schematic diagram of a system architecture for a text classification method provided in an embodiment of this application. The system architecture includes a terminal 101 and a server 102. The terminal 101 sends the text to be classified to the server 102 so that the server 102 can perform text classification on the text to be classified.

[0118] The terminal 101 includes, but is not limited to, smartphones, tablets, laptops, desktop computers, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, and extended reality devices (augmented reality devices, virtual reality devices, mixed reality devices), but is not limited to these. The server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal 101 and server 102 can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions. For example, the terminal 101 and server 102 can be connected via a network, which can be wired or wireless.

[0119] Terminal 101 collects multiple texts to be classified and sends them to server 102. Server 102 uses a preset classification model to predict the category of each text to be classified, obtaining the first distribution probability of each text to belong to multiple preset categories. The preset classification model is obtained by server 102 through knowledge distillation-based model training of a first classification model and a second classification model based on multiple first training texts and the category label data annotated in each first training text. The model complexity of the first classification model is greater than that of the second classification model.

[0120] As an example, in a payment-related business complaint scenario, multiple first training texts include "historical complaint text 1", "historical complaint text 2", ... and "historical complaint text m", where m is a positive integer and m≥500; the first classification model is "Model A" with higher model complexity, and the second classification model is "Model B" with lower model complexity; multiple texts to be classified include "current complaint text 1", "current complaint text 2", ... and "current complaint text n", where n is a positive integer and n>3; multiple preset categories include "consumer complaint", "payer complaint", and "payment fee complaint". Server 102 pre-trains "Model A" and "Model B" using knowledge distillation-based model training data of "historical complaint text 1", "historical complaint text 2", ... and "historical complaint text m" and their labeled category data to obtain a preset classification model; Server 102 inputs "current complaint text i" into the preset classification model to predict the category, and outputs the first distribution probability of "current complaint text i" belonging to "consumer complaint category", "fee collector complaint category" and "payment fee complaint category" as "distribution probability 1i", where i is a positive integer, i = 1, 2, ..., n.

[0121] Server 102 performs clustering processing on multiple texts to be classified, obtaining the second distribution probability of each text belonging to multiple preset categories. As an example, based on the above example, server 102 clusters "current complaint text 1", "current complaint text 2", ... and "current complaint text n", and obtains the second distribution probability of "current complaint text i" belonging to "consumer complaint category", "payer complaint category" and "payment fee complaint category" as "distribution probability 2i".

[0122] For each text to be classified, server 102 fuses the first distribution probability and the second distribution probability of the text to be classified belonging to multiple preset categories to obtain the fused distribution probability of the text to be classified belonging to multiple preset categories. As an example, based on the above example, for "current complaint text i", "distribution probability 1i" and "distribution probability 2i" are fused to obtain the fused distribution probability of "current complaint text i" belonging to "consumer complaint category", "payer complaint category" and "payment fee complaint category" as "distribution probability i".

[0123] Server 102 determines the classification category of the text to be classified based on the preset category corresponding to the maximum probability component in the fused distribution probability. As an example, based on the above example, server 102 takes the preset category corresponding to the maximum probability component in "distribution probability i" as the classification category of "current complaint text i"; if "distribution probability i" is specifically "[0.1, 0.1, 0.8]", since the preset category corresponding to the maximum probability component "0.8" is "payment fee complaint", then the classification category of "current complaint text i" is "payment fee complaint".

[0124] In other words, a pre-defined classification model is obtained by pre-training a first classification model and a second classification model using knowledge distillation, using multiple first training texts and the category labels labeled for each first training text. The first classification model has a higher model complexity than the second classification model. Each text to be classified is then input into the pre-defined classification model for category prediction, outputting the first probability distribution of each text belonging to multiple pre-defined categories. Because the knowledge distillation-based model training method reduces model complexity without compromising classification accuracy, the pre-defined classification model can, to some extent, guarantee the classification performance of the texts to be classified. Furthermore, clustering is performed on the multiple texts to be classified to obtain the second probability distribution of each text belonging to multiple pre-defined categories. This clustering method compensates for the weak classification ability of the pre-defined classification model caused by a small total number of first training texts or class imbalance. For each text to be classified, the first and second probability distributions of the text belonging to multiple pre-defined categories are fused to obtain the fused probability distribution of the text belonging to multiple pre-defined categories. This fusion method makes the fused probability distribution more accurate than the first and second probability distributions. Using the preset category corresponding to the highest probability component in the fusion distribution probability as the classification category of the text to be classified can improve the classification accuracy of the text to be classified, thereby improving the classification effect of the text to be classified.

[0125] The text classification method provided in this application relates to Artificial Intelligence (AI) technology. AI refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines capable of reacting in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0126] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics.

[0127] The text classification method provided in this application mainly involves major areas of artificial intelligence technology, including Natural Language Processing (NLP) and Machine Learning (ML) / Deep Learning. Natural Language Processing is a crucial area within computer science and artificial intelligence. It studies various theories and methods that enable effective communication between humans and computers using natural language. Natural Language Processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language, i.e., the language people use in daily life, and thus it has a close relationship with linguistic research. Natural Language Processing technologies typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0128] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0129] The embodiments of this application can be applied to various scenarios, including but not limited to cloud technology scenarios, artificial intelligence scenarios, smart transportation scenarios, assisted driving scenarios, map scenarios, and vehicle scenarios.

[0130] Next, the text classification method provided in the embodiments of this application will be described in detail with reference to the accompanying drawings. See also Figure 2 , Figure 2 A flowchart of a text classification method provided in this application embodiment, the method including:

[0131] S201: Predict the category of each text to be classified among multiple texts by using a preset classification model, and obtain the first distribution probability of each text to be classified belonging to multiple preset categories; the preset classification model is obtained by training the first classification model and the second classification model based on knowledge distillation based on multiple first training texts and the category label data annotated in each first training text. The model complexity of the first classification model is greater than that of the second classification model.

[0132] In related technologies, a deep learning model is pre-trained using training text with category labels to obtain a text classification model. The text to be classified is then input into the text classification model for category prediction, and the predicted category to which the text belongs is taken as the classification category of the text. When the complexity of the deep learning model is high, the deployment requirements of the text classification model are demanding; when the complexity of the deep learning model is low, the classification accuracy of the text classification model is low. In other words, both high and low complexity of the deep learning model lead to poor classification performance of the text to be classified.

[0133] Therefore, in this embodiment, to balance model deployment performance and model classification accuracy, the knowledge learned by the more complex first classification model can be transferred to the less complex second classification model to obtain a preset classification model that reduces model complexity while maintaining classification accuracy as much as possible. Specifically, based on multiple first training texts and the category label data annotated for each first training text, the first and second classification models are trained using knowledge distillation to obtain the preset classification model. Based on this, each text to be classified is input into the preset classification model for category prediction, and the probability distribution of each text to be classified into multiple preset categories is output, denoted as the first distribution probability of each text to be classified into multiple preset categories. This prediction method can, to a certain extent, guarantee the classification performance of the texts to be classified.

[0134] Prior to the implementation of S201, the specific training method of the pre-defined classification model is as follows: For each first training text, firstly, the first training text is input into the first pre-trained model with higher model complexity in the first classification model to predict the category, and the probability distribution of the first training text to belong to multiple pre-defined categories is obtained, which is denoted as the third distribution probability of each text to be classified to belong to multiple pre-defined categories; next, instead of directly normalizing the third distribution probability, distillation is performed before normalization, that is, the third distribution probability is input into the normalization layer in the first classification model to perform knowledge distillation-based normalization to obtain the target distribution probability of the first training text to belong to multiple pre-defined categories.

[0135] Secondly, the first training text is input into the second pre-trained model with lower model complexity in the second classification model for category prediction, obtaining the probability distribution of the first training text to belong to multiple preset categories, denoted as the fourth distribution probability of each text to be classified to belong to multiple preset categories. Next, on the one hand, distillation is performed before normalizing the fourth distribution probability, that is, the fourth distribution probability is input into the first normalization layer in the second classification model for knowledge distillation-based normalization, obtaining the first prediction probability of the first training text to belong to multiple preset categories; on the other hand, the fourth distribution probability is directly normalized, that is, the fourth distribution probability is input into the second normalization layer in the second classification model for normalization, obtaining the second prediction probability of the first training text to belong to multiple preset categories.

[0136] Then, considering the transfer of knowledge learned by the more complex first classification model to the less complex second classification model, the target distribution probability of the first training text to the multiple preset categories is actually used to train the first prediction probability of the first training text to the multiple preset categories, while the category label data of the first training text is actually used to train the second prediction probability of the first training text to the multiple preset categories. Therefore, by combining the first prediction probability, the target distribution probability, the second prediction probability and the category label data with the preset loss function of the second classification model, the model parameters of the second classification model can be trained until the model converges.

[0137] Finally, by selecting only the second pre-trained model and the second normalization layer from the trained second classification model, a preset classification model can be obtained that minimizes model complexity while maintaining classification accuracy. Therefore, this application provides a possible implementation method, and the training steps of the preset classification model may include, for example, the following S1-S7:

[0138] S1: For each first training text, the first pre-trained model in the first classification model is used to predict the category of the first training text, and the third distribution probability of the first training text belonging to multiple preset categories is obtained.

[0139] Common text classification models include Long Short-Term Memory (LSTM) networks, text Convolutional Neural Networks (textCNN), and Bidirectional Encoder Representations from Transformer (BERT). See also Figure 3 , Figure 3This application provides a schematic diagram of LSTM-based text classification; wherein, multiple word segmentation vectors w1, w2, ..., w... n-1 and w n The input feature extraction layer yields multiple hidden layer vectors h1, h2, ..., h3. n-1 and h n h1, h2, ..., h n-1 and h n The text feature vector is obtained by inputting into the self-attention layer, and h1, h2, ..., h... n-1 and h n And text category label data [c1, c2, ..., c k The input label attention layer obtains the category feature vector. The text feature vector and the category feature vector are input into the adaptive fusion layer to obtain the fused feature vector. The fused feature vector is then used by a multi-layer perceptron (MLP) to predict the category of the text and obtain the probability distribution of the multiple categories to which the text belongs [p1, p2, ..., p]. k ].

[0140] See Figure 4 , Figure 4 This is a schematic diagram of text classification based on textCNN provided in an embodiment of this application; wherein, the 6×5 text matrix of the text "Today the weather is very nice, let's go out and play" is convolved with two convolution sizes (3×5, 4×5), each convolution size has two convolution kernels, for a total of four convolution kernels, and the feature map of each convolution size is obtained by combining the activation function, each convolution size has two feature maps, for a total of four feature maps; each feature map is max pooled to obtain four single vectors and concatenated into one one-dimensional vector, and the one-dimensional vector is normalized to obtain the probability distribution of multiple categories to which the text belongs [1, -1].

[0141] See Figure 5 , Figure 5 This is a schematic diagram of a BERT-based text classification method provided in an embodiment of this application; wherein, the text [CLS] [SEP] is input into the pre-trained model BERT as input content to obtain an encoding vector including the [CLS] encoding vector, and the probability distribution of multiple categories to which the text belongs is obtained by classifying the [CLS] encoding vector.

[0142] Based on the above explanation, BERT was tested on four commonly used standard text classification datasets. BERT's classification performance surpassed that of LSTM or textCNN. Therefore, the first pre-trained model in the first classification model can be BERT, which has a higher model complexity.

[0143] Furthermore, before implementing S1, it is necessary to pre-acquire multiple first training texts and category label data for each first training text. The multiple first training texts can be obtained by randomly sampling multiple collected historical texts. Considering that the prerequisite for the preset classification model to learn classification ability is the need for a certain number of training texts with category label data, the total number of multiple first training texts needs to be greater than or equal to a certain number, for example, a first quantity. The category label data for each first training text is obtained by the first object labeling each first training text according to the preset classification business requirements corresponding to the multiple first training texts. Therefore, this application provides a possible implementation method. The steps for obtaining multiple first training texts can be, for example, as follows: randomly sampling multiple collected historical texts to obtain multiple first training texts; the total number of multiple first training texts is greater than or equal to the first quantity. The steps for obtaining the category label data for each first training text can be, for example, as follows: in response to the first object's operation of labeling each first training text according to the preset classification business requirements corresponding to the multiple first training texts, the category label data for each first training text is obtained.

[0144] As an example, in a payment-related business complaint scenario, multiple historical texts include "Historical Complaint Text 1", "Historical Complaint Text 2", ... and "Historical Complaint Text 10000", with a first quantity of 500. The first object is "Object A", and the preset classification business requirement is "Payment-related Business Complaint Requirement". First, "Historical Complaint Text 1", "Historical Complaint Text 2", ... and "Historical Complaint Text 10000" are randomly sampled to obtain multiple first training texts including "Historical Complaint Text 1", "Historical Complaint Text 2", ... and "Historical Complaint Text m", where m is a positive integer, m≥500. Then, in response to "Object A" classifying "Historical Complaint Text j" according to "Payment-related Business Complaint Requirement", the category label data for "Historical Complaint Text j" is obtained, where j is a positive integer, j=1,2,...,m. When there are multiple preset categories including "Consumer Complaints", "Payer Complaints", and "Payment Fee Complaints", if "Historical Complaint Text j" belongs to "Consumer Complaints", the category label data of "Historical Complaint Text j" is "[1, 0, 0]"; if "Historical Complaint Text j" belongs to "Payer Complaints", the category label data of "Historical Complaint Text j" is "[0, 1, 0]"; if "Historical Complaint Text j" belongs to "Payment Fee Complaints", the category label data of "Historical Complaint Text j" is "[0, 0, 1]".

[0145] S2: By using the normalization layer in the first classification model, the third distribution probability is normalized based on knowledge distillation to obtain the target distribution probability of the first training text belonging to multiple preset categories.

[0146] The normalization process for the third distribution probability based on knowledge distillation refers to: first, distilling the third distribution probability, for example, dividing the third distribution probability by T; and then normalizing the distilled third distribution probability.

[0147] S3: Using the second pre-trained model in the second classification model, predict the category of each first training text to obtain the fourth distribution probability of the multiple preset categories to which the first training text belongs; the model complexity of the first pre-trained model is greater than that of the second pre-trained model.

[0148] In addition, considering that BERT's classification performance can surpass that of LSTM or textCNN, the second pre-trained model in the second classification model can be BERT, which has lower model complexity.

[0149] S4: Through the first normalization layer in the second classification model, the fourth distribution probability is normalized based on knowledge distillation to obtain the first prediction probability of the first training text belonging to multiple preset categories.

[0150] The normalization process for the fourth distribution probability based on knowledge distillation refers to: first, distilling the fourth distribution probability, for example, dividing the fourth distribution probability by T; and then normalizing the distilled fourth distribution probability.

[0151] S5: The fourth distribution probability is normalized by the second normalization layer in the second classification model to obtain the second prediction probability of the first training text belonging to multiple preset categories.

[0152] S6: Based on the first predicted probability, the target distribution probability, the second predicted probability, the category label data of the first training text annotation, and the preset loss function of the second classification model, train the model parameters of the second classification model until the model converges.

[0153] In the specific implementation of S6, considering that the target distribution probability of the first training text to which it belongs is actually used to train the first predicted probability of the first training text to which it belongs, and the category label data of the first training text is actually used to train the second predicted probability of the first training text to which it belongs, the preset loss function of the second classification model is set as two loss functions: a first loss function for calculating the loss between the first predicted probability and the target distribution probability, and a second loss function for calculating the loss between the second predicted probability and the category label data. Based on this, firstly, the first predicted probability and the target distribution probability are substituted into the first loss function to calculate the first loss corresponding to the first training text, and the second predicted probability and the category label data are substituted into the second loss function to calculate the second loss corresponding to the first training text; then, third weights and fourth weights are assigned respectively to adjust the ratio of the first loss and the second loss to obtain the total loss, where the sum of the third weight and the fourth weight is 1; finally, the model parameters of the second classification model are iteratively trained through the total loss until the model converges. Model convergence is manifested by the total loss no longer decreasing or reaching the preset number of iterations. Therefore, this application provides a possible implementation, wherein the preset loss function includes a first loss function and a second loss function; S6 may include, for example, the following S61-S64:

[0154] S61: Determine the first loss corresponding to the first training text based on the first predicted probability, the target distribution probability, and the first loss function.

[0155] S62: Determine the second loss corresponding to the first training text based on the second predicted probability, the category label data, and the second loss function.

[0156] S63: Weight the first loss, the third weight corresponding to the first loss, the second loss, and the fourth weight corresponding to the second loss to obtain the total loss corresponding to the first training text; the sum of the third weight and the fourth weight is 1.

[0157] S64: Train the model parameters of the second classification model based on the total loss until the model converges.

[0158] As an example, see Figure 6 , Figure 6This application provides a schematic diagram of model training based on knowledge distillation for a first classification model and a second classification model; wherein, the first classification model is "Model A" with higher model complexity, the second classification model is "Model B" with lower model complexity, the first pre-trained model is "First BERT" with higher model complexity, the second pre-trained model is "Second BERT" with higher model complexity, and multiple preset categories include "Consumer Complaints", "Payers' Complaints", and "Payment Fee Complaints". Based on the above example, firstly, the first training texts "historical complaint text 1", "historical complaint text 2", ... and "historical complaint text m" and "historical complaint text j" are input into the "first BERT" in "model A" for category prediction, to obtain the third distribution probability of "historical complaint text j" belonging to the "consumer complaint", "fee collector complaint", and "payment fee complaint" categories. The third distribution probability is input into the normalization layer in "model A" and first distilled by dividing by T and then normalized to obtain the target distribution probability of "historical complaint text j" belonging to the "consumer complaint", "fee collector complaint", and "payment fee complaint" categories as "[0.2, 0.2, 0.6]".

[0159] Secondly, the "historical complaint text j" is input into the "second BERT" in "Model B" for category prediction, obtaining the fourth distribution probability of the "historical complaint text j" belonging to the "consumer complaint", "charger complaint", and "payment fee complaint" categories. Not only is the fourth distribution probability input into the first normalization layer in "Model B" first distilled by dividing by T and then normalized to obtain the first predicted probability of the "historical complaint text j" belonging to the "consumer complaint", "charger complaint", and "payment fee complaint" categories, but the fourth distribution probability is also directly input into the second normalization layer in "Model B" for normalization to obtain the second predicted probability of the "historical complaint text j" belonging to the "consumer complaint", "charger complaint", and "payment fee complaint" categories.

[0160] Then, by using the first predicted probability, "[0.2, 0.2, 0.6]", the second predicted probability, and the category label data "[0, 0, 1]" of "historical complaint text j", combined with the first and second loss functions of "model B", the model parameters of "model B" can be trained until the model converges; where the first and second loss functions are both cross-entropy loss functions.

[0161] S7: Determine the second pre-trained model and the second normalization layer in the trained second classification model as the preset classification model.

[0162] As an example, see Figure 7 , Figure 7A schematic diagram of a preset classification model provided in this application embodiment; wherein, based on the above example, the above... Figure 6 In "Model B", the "Second BERT" and the second normalization layer are used as the preset classification model. Based on this, multiple texts to be classified include "Current Complaint Text 1", "Current Complaint Text 2", ... and "Current Complaint Text n", where n is a positive integer. When n > 3, "Current Complaint Text i" is input into the preset classification model for category prediction. The first distribution probability of "Current Complaint Text i" belonging to the "Consumer Complaint", "Payer Complaint", and "Payment Fee Complaint" categories is output as "Distribution Probability 1i", where i is a positive integer, i = 1, 2, ..., n.

[0163] S202: Perform clustering processing on multiple texts to be classified to obtain the second distribution probability of each text to be classified belonging to multiple preset categories.

[0164] In related technologies, when the total number of training texts is small or the categories are imbalanced, the classification ability of the text classification model is weak. Relying solely on the text classification model to predict the category of the text to be classified can easily lead to low classification accuracy, resulting in poor classification performance of the text to be classified.

[0165] Therefore, in this embodiment of the application, after performing S201 to predict the category of each text to be classified according to the preset classification model and obtain the first distribution probability of each text to be classified belonging to multiple preset categories, in order to make up for the shortcomings of the weak classification ability of the preset classification model caused by the small total number of multiple first training texts or the imbalance of categories, it is also necessary to cluster multiple texts to be classified to obtain the probability distribution of each text to be classified belonging to multiple preset categories, which is denoted as the second distribution probability of each text to be classified belonging to multiple preset categories.

[0166] In the specific implementation of S202, given multiple preset categories, firstly, multiple initial centroids are randomly selected, the same number as the number of preset categories. Secondly, the distance between each text to be classified and each initial centroid is calculated, and each text to be classified is clustered to the initial centroid closest to it. Then, the multiple texts to be classified under the preset category corresponding to each initial centroid represent a cluster. Then, after all the texts to be classified have been clustered, each initial centroid is recalculated according to the multiple texts to be classified under its corresponding preset category to update the initial centroid. The above process is repeated until the clustering converges. Clustering convergence means that no (or the minimum number) texts to be classified are re-clustered to different initial centroids, no (or the minimum number) initial centroids change, and the sum of squared distance errors between the multiple texts to be classified under the preset category corresponding to each initial centroid and that initial centroid is locally minimized. Finally, based on the preset category to which each text to be classified belongs after clustering convergence, the second distribution probability of the multiple preset categories to which each text to be classified belongs can be determined. Therefore, this application provides a possible implementation, and S202 may include, for example, the following S2021-S2025:

[0167] S2021: Determine multiple initial center points corresponding to multiple preset categories.

[0168] S2022: Cluster multiple texts to be classified based on the distance between each text to be classified and each initial centroid, and determine the preset category to which each text to be classified belongs.

[0169] S2023: Update each initial centroid based on the multiple unclassified texts clustered under the preset category corresponding to each initial centroid.

[0170] S2024: Repeat the clustering process for multiple texts to be classified based on the distance between each text to be classified and each initial centroid until the clustering converges.

[0171] S2025: Based on the preset category to which each text to be classified belongs after clustering convergence, determine the second distribution probability of multiple preset categories to which each text to be classified belongs.

[0172] As an example, based on the above example, based on multiple unclassified texts "Current Complaint Text 1", "Current Complaint Text 2", ... and "Current Complaint Text n", three initial centroids corresponding to multiple "Consumer Complaint Class", "Payer Complaint Class", and "Payment Fee Complaint Class" are determined; the distance between "Current Complaint Text i" and each initial centroid is calculated, and "Current Complaint Text i" is clustered to the initial centroid closest to it; after "Current Complaint Text 1", "Current Complaint Text 2", ... and "Current Complaint Text n" have all been clustered, the multiple unclassified texts clustered under their corresponding preset categories under each initial centroid are recalculated to update the initial centroids; the above process is repeated until the clustering converges, and based on the preset category to which "Current Complaint Text i" belongs after the clustering converges, the second distribution probability of "Current Complaint Text i" belonging to the "Consumer Complaint Class", "Payer Complaint Class", and "Payment Fee Complaint Class" is determined as "Distribution Probability 2i".

[0173] S203: For each text to be classified, the first distribution probability of the text to be classified belonging to multiple preset categories and the second distribution probability of the text to be classified belonging to multiple preset categories are fused to obtain the fused distribution probability of the text to be classified belonging to multiple preset categories.

[0174] In this embodiment of the application, after executing S201-S202 to obtain the first distribution probability of each text to be classified belonging to multiple preset categories predicted by the preset classification model and the second distribution probability of each text to be classified belonging to multiple preset categories determined by clustering, in order to make the probability distribution of each text to be classified to multiple preset categories more accurate, for each text to be classified, the first distribution probability and the second distribution probability are processed by a fusion method to obtain the fused distribution probability of the text to be classified to multiple preset categories. This fused distribution probability is more accurate than the first distribution probability and the second distribution probability.

[0175] In the specific implementation of S203, considering the different importance of the first distribution probability and the second distribution probability, a first weight and a second weight are assigned to the first distribution probability and the second distribution probability, respectively. These first weights and second weights are used to adjust the ratio of the first distribution probability and the second distribution probability to obtain a weighted fused distribution probability; wherein the sum of the first weight and the second weight is 1. Therefore, this application provides a possible implementation, and S203 may include, for example, the following S2031-S2032:

[0176] S2031: Determine the first weight corresponding to the first probability distribution and the second weight corresponding to the second probability distribution; the sum of the first weight and the second weight is 1;

[0177] S2032: The first distribution probability and first weight of the multiple preset categories to which the text to be classified belongs, the second distribution probability and second weight of the multiple preset categories to which the text to be classified belongs, are weighted to obtain the fused distribution probability of the multiple preset categories to which the text to be classified belongs.

[0178] As an example, the first weight is α, the second weight is β, and α+β=1. Based on the above example, for each text to be classified, “current complaint text i”, the weights “distribution probability 1i”, α, “distribution probability 2i” and β are added to obtain the fusion distribution probability of the multiple preset categories “consumer complaint category”, “fee collector complaint category” and “payment fee complaint category” to which “current complaint text i” belongs, which is “distribution probability i”, that is, “distribution probability i” = αדdistribution probability 1i”+βדdistribution probability 2i”.

[0179] S204: Determine the classification category of the text to be classified based on the preset category corresponding to the highest probability component in the fusion distribution probability.

[0180] In this embodiment of the application, after executing S203 to obtain the fusion distribution probability of the multiple preset categories to which the text to be classified belongs, and considering that the fusion distribution probability is more accurate than the first distribution probability and the second distribution probability, and considering that the fusion distribution probability can more accurately represent the probability distribution of the multiple preset categories to which the text to be classified belongs, the preset category corresponding to the largest probability component in the fusion distribution probability can be directly used as the classification category to which the text to be classified belongs, so as to improve the classification accuracy of the text to be classified and thus improve the classification effect of the text to be classified.

[0181] As an example, based on the above example, when the fusion distribution probability of the multiple preset categories "Consumer Complaints", "Payers' Complaints" and "Payment Fee Complaints" to which the text to be classified "current complaint text i" belongs is "distribution probability i", the preset category corresponding to the highest probability component in "distribution probability i" is taken as the classification category to which "current complaint text i" belongs. If the "distribution probability i" is specifically "[0.8, 0.1, 0.1]", then the category to which "current complaint text i" belongs is "consumer complaint" because the preset category corresponding to the maximum probability component "0.8" is "consumer complaint". If the "distribution probability i" is specifically "[0.1, 0.8, 0.1]", then the category to which "current complaint text i" belongs is "payer complaint" because the preset category corresponding to the maximum probability component "0.8" is "payment fee complaint". If the "distribution probability i" is specifically "[0.1, 0.1, 0.8]", then the category to which "current complaint text i" belongs is "payment fee complaint" because the preset category corresponding to the maximum probability component "0.8" is "payment fee complaint".

[0182] Furthermore, in this embodiment, after obtaining the category label data of each first training text as described above, before S1 is implemented, since the category label data of each first training text is obtained by the first object according to the preset classification business requirements, there may be a problem that the accuracy of the category label data of some first training texts is low. To solve this problem, firstly, multiple texts to be annotated can be randomly sampled from multiple first training texts, the total number of which is greater than or equal to a second number, the second number being less than the aforementioned first number; then, the annotation category data of each text to be annotated is obtained by the second object, which is different from the first object, according to the preset classification business requirements; finally, the multiple category label data corresponding to the multiple texts to be annotated are determined to be consistent with the multiple texts to be annotated. If the consistency level among multiple annotation category data is greater than or equal to the preset consistency level, it means that different objects have consistent category labels for each text to be annotated according to the preset classification business requirements. In this case, the accuracy of the category label data of each first training text is high, that is, the category label data of each first training text is correct. If not, it means that different objects have inconsistent category labels for each text to be annotated according to the preset classification business requirements. In this case, the labeling accuracy of the category label data of each first training text is low. It is necessary to update the category label data of each first training text to improve the accuracy of the category label data of each first training text. This will facilitate the subsequent execution of S6 based on the updated category label data of each first training text, further improving the classification accuracy of the preset classification model.

[0183] Therefore, this application provides a possible implementation, and the method may further include the following steps: randomly sampling multiple first training texts to obtain multiple texts to be annotated; the total number of multiple texts to be annotated is greater than or equal to a second number, and the second number is less than the first number; in response to the second object performing a category labeling operation on each text to be annotated according to a preset classification business requirement, obtaining annotation category data for each text to be annotated; if the consistency between the multiple category label data corresponding to the multiple texts to be annotated and the multiple annotation category data corresponding to the multiple texts to be annotated is greater than or equal to a preset consistency level, determining that the category label data of each first training text is correctly labeled; if the consistency level is less than the preset consistency level, updating the category label data of each first training text.

[0184] Furthermore, in this embodiment, after S1-S7 and before S201, the classification accuracy of the preset classification model can be verified. Specifically, based on multiple test texts different from the multiple first training texts and the category label data of each test text, firstly, each test text is input into the preset classification model for category prediction, outputting the probability distribution of multiple preset categories to which each test text belongs, denoted as the third predicted probability of each test text belonging to multiple preset categories; then, the third predicted probability of each test text belonging to multiple preset categories and the category label data of each test text are substituted into preset verification indicators to verify the preset classification model. The model validation results of the preset classification model are obtained. Finally, it is determined whether the model validation results meet the preset validation conditions. If yes, it indicates that the classification accuracy of the preset classification model is high, and S201 can be executed directly. If no, it indicates that the classification accuracy of the preset classification model is low, and it is necessary to extract multiple second training texts from multiple historical texts to update multiple first training texts. Based on the multiple second training texts and the category label data annotated in each second training text, the first and second classification models are retrained using knowledge distillation to update the preset classification model, improve the classification accuracy of the updated preset classification model, and facilitate subsequent execution of S201 based on the updated preset classification model. Therefore, this application provides a possible implementation method, which may further include the following S8-S:

[0185] S8: Using a pre-defined classification model, predict the category of each test text among multiple test texts that are different from the multiple first training texts, and obtain the third prediction probability of each test text belonging to multiple pre-defined categories.

[0186] S9: Based on the third predicted probability of each test text belonging to multiple preset categories, the category label data of each test text, and the preset verification index, perform model verification on the preset classification model and obtain the model verification result of the preset classification model.

[0187] The preset validation metric can be any one of the common multi-class classification model evaluation metrics such as confusion matrix, accuracy, and recall. Therefore, this application provides a possible implementation method in which the preset validation metric includes any one of confusion matrix, accuracy, and recall.

[0188] As an example, see Figure 8 , Figure 8This is a schematic diagram of a confusion matrix provided in an embodiment of this application. The vertical labels of the confusion matrix represent the true categories corresponding to the category label data of multiple test samples, from top to bottom: "Consumer Complaints," "Payers' Complaints," and "Payment Fee Complaints." The horizontal labels of the confusion matrix represent the predicted categories corresponding to the third predicted probabilities of multiple test samples, from left to right: "Consumer Complaints," "Payers' Complaints," and "Payment Fee Complaints." Based on this, the first row and first column of the confusion matrix represent the number of test samples that are actually in the "Consumer Complaints" category and predicted to belong to it, which is "13." The first row and second column represent the number of test samples that are actually in the "Consumer Complaints" category and predicted to belong to it, which is "0." Similarly, the third row and third column represent the number of test samples that are actually in the "Payment Fee Complaints" category and predicted to belong to it, which is "9."

[0189] S10: If the model validation result does not meet the preset validation conditions, update multiple first training texts based on multiple historical texts to obtain multiple second training texts.

[0190] S11: Based on multiple second training texts and the category label data annotated for each second training text, perform knowledge distillation-based model training on the first classification model and the second classification model, and update the preset classification model.

[0191] Furthermore, in the embodiments of this application, in the text classification reply scenario, the purpose of classifying the text to be classified may be to find a reply text that is more suitable for the classification category of the text to be classified, so as to reply to the text to be classified more accurately. Therefore, this application provides a possible implementation method. When the text classification method is applied to the text classification reply scenario, the method may further include, for example, the following S12: determining the reply text that matches the classification category of the text to be classified according to the classification category of the text to be classified.

[0192] As an example, based on the above example, when the text classification method is applied to a text classification response scenario, if the category to which the text to be classified, "current complaint text i," belongs is "consumer complaint," then the text to be replied to that "current complaint text i" that matches "consumer complaint" is designated as "reply text a"; if the category to which the text to be classified, "current complaint text i," belongs is "payer complaint," then the text to be replied to that "current complaint text i" that matches "payer complaint" is designated as "reply text b"; if the category to which the text to be classified, "current complaint text i," belongs is "payment fee complaint," then the text to be replied to that "current complaint text i" that matches "payment fee complaint" is designated as "reply text c."

[0193] As can be seen from the above technical solution, a pre-defined classification model is obtained by pre-training the first and second classification models using knowledge distillation based on multiple first training texts and the category labels labeled for each first training text. Since the complexity of the first classification model is greater than that of the second classification model, each text to be classified is input into the pre-defined classification model for category prediction, outputting the first distribution probability of each text belonging to multiple pre-defined categories. Because the knowledge distillation-based model training method reduces model complexity while minimizing the reduction in classification accuracy, the pre-defined classification model can, to a certain extent, guarantee the classification performance of the texts to be classified. Furthermore, clustering the multiple texts to be classified yields the second distribution probability of each text belonging to multiple pre-defined categories. This clustering method can compensate for the weakness of the pre-defined classification model caused by a small total number of first training texts or an imbalanced category distribution. For each text to be classified, the first probability distribution and the second probability distribution of the text to be classified belonging to multiple preset categories are fused to obtain the fused probability distribution of the text to be classified belonging to multiple preset categories. This fusion method makes the fused probability distribution more accurate than the first and second probability distributions. Taking the preset category corresponding to the highest probability component in the fused probability distribution as the classification category of the text to be classified can improve the classification accuracy of the text to be classified, thereby improving the classification effect.

[0194] Next, in the context of payment-related business complaints, the text classification method provided in this application embodiment will be described in detail with reference to the accompanying drawings. See also Figure 9 , Figure 9 A flowchart illustrating a text classification method tailored to a practical application scenario, provided in this application embodiment, includes:

[0195] S901: The category of each current complaint text in multiple current complaint texts is predicted by a preset classification model to obtain the first distribution probability of each current complaint text belonging to multiple payment complaint categories; the preset classification model is obtained by training the first classification model and the second classification model based on knowledge distillation based on multiple historical complaint texts and the category label data marked on each historical complaint text. The model complexity of the first classification model is greater than that of the second classification model.

[0196] S902: Cluster multiple current complaint texts to obtain the second distribution probability of each current complaint text belonging to multiple payment complaint categories.

[0197] S903: Determine the first weight corresponding to the first probability distribution and the second weight corresponding to the second probability distribution; the sum of the first weight and the second weight is 1.

[0198] S904: For each current complaint text, the first distribution probability, first weight, second distribution probability, and second weight of the multiple payment complaint categories to which the current complaint text belongs are weighted to obtain the fusion distribution probability of the multiple payment complaint categories to which the current complaint text belongs.

[0199] S905: Determine the classification category of the current complaint text based on the payment complaint category corresponding to the highest probability component in the fusion distribution probability.

[0200] S906: Based on the category to which the current complaint text belongs, determine the corresponding text to be replied to that matches the category.

[0201] As can be seen from the above technical solutions, the knowledge distillation-based model training method reduces model complexity while minimizing the reduction in classification accuracy. Therefore, the pre-defined classification model can, to a certain extent, guarantee the classification performance of the current complaint text. Furthermore, the clustering method can compensate for the weakness of the pre-defined classification model caused by a small total number of historical complaint texts or class imbalance. The fusion method makes the fused distribution probability more accurate compared to the first and second distribution probabilities. Based on the pre-defined category corresponding to the maximum probability component in the fused distribution probability, the classification category of the current complaint text is determined, improving the classification accuracy of the current complaint text and thus enhancing the accuracy of the response text for the current complaint text.

[0202] It should be noted that, based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods.

[0203] Next, based on Figure 2 Corresponding to the text classification method provided in the embodiments, this application also provides a text classification device, see below. Figure 10 , Figure 10 This is a structural diagram of a text classification device provided in an embodiment of the present application. The text classification device 1000 includes: a prediction unit 1001, a clustering unit 1002, a fusion unit 1003, and a determination unit 1004.

[0204] The prediction unit 1001 is used to predict the category of each text to be classified among multiple texts to be classified by using a preset classification model, and to obtain the first distribution probability of each text to be classified belonging to multiple preset categories. The preset classification model is obtained by training the first classification model and the second classification model based on knowledge distillation based on multiple first training texts and the category label data annotated in each first training text. The model complexity of the first classification model is greater than that of the second classification model.

[0205] Clustering unit 1002 is used to perform clustering processing on multiple texts to be classified, and to obtain the second distribution probability of multiple preset categories to which each text belongs;

[0206] The fusion unit 1003 is used to fuse the first distribution probability of the multiple preset categories to which the text to be classified belongs and the second distribution probability of the multiple preset categories to which the text to be classified belongs for each text to be classified, so as to obtain the fused distribution probability of the multiple preset categories to which the text to be classified belongs.

[0207] The determining unit 1004 is used to determine the classification category to which the text to be classified belongs based on the preset category corresponding to the maximum probability component in the fusion distribution probability.

[0208] In one possible implementation, the fusion unit 1003 is specifically used for:

[0209] Determine the first weight corresponding to the first probability distribution and the second weight corresponding to the second probability distribution; the sum of the first weight and the second weight is 1;

[0210] The first distribution probability and first weight of the multiple preset categories to which the text to be classified belongs, the second distribution probability and second weight of the multiple preset categories to which the text to be classified belongs, are weighted to obtain the fused distribution probability of the multiple preset categories to which the text to be classified belongs.

[0211] In one possible implementation, the device further includes: a training unit;

[0212] The training unit is specifically used to predict the category of each first training text by using the first pre-trained model in the first classification model, and to obtain the third distribution probability of the first training text belonging to multiple preset categories.

[0213] By using the normalization layer in the first classification model, the third distribution probability is normalized based on knowledge distillation to obtain the target distribution probability of the first training text belonging to multiple preset categories.

[0214] The second pre-trained model in the second classification model is used to predict the category of each first training text, and obtain the fourth distribution probability of the first training text belonging to multiple preset categories; the model complexity of the first pre-trained model is greater than that of the second pre-trained model.

[0215] By using the first normalization layer in the second classification model, the fourth distribution probability is normalized based on knowledge distillation to obtain the first predicted probability of the first training text belonging to multiple preset categories.

[0216] The fourth distribution probability is normalized by the second normalization layer in the second classification model to obtain the second predicted probability of the first training text belonging to multiple preset categories.

[0217] Based on the first predicted probability, the target distribution probability, the second predicted probability, the category label data of the first training text annotation, and the preset loss function of the second classification model, train the model parameters of the second classification model until the model converges.

[0218] The second pre-trained model and the second normalization layer in the trained second classification model are determined as the preset classification model.

[0219] In one possible implementation, the training unit is specifically used for:

[0220] Based on the first prediction probability, the target distribution probability, and the first loss function, determine the first loss corresponding to the first training text;

[0221] Based on the second predicted probability, the category label data, and the second loss function, determine the second loss corresponding to the first training text;

[0222] The first loss, the third weight corresponding to the first loss, the second loss, and the fourth weight corresponding to the second loss are weighted to obtain the total loss corresponding to the first training text; the sum of the third weight and the fourth weight is 1.

[0223] Train the parameters of the second classification model based on the total loss until the model converges.

[0224] In one possible implementation, the apparatus further includes: a sampling unit;

[0225] The sampling unit is used to randomly sample multiple historical texts collected to obtain multiple first training texts; the total number of multiple first training texts is greater than or equal to the first number.

[0226] In one possible implementation, the apparatus further includes: a labeling unit;

[0227] The annotation unit is used to respond to the first object's category annotation operation for each first training text according to the preset classification business requirements corresponding to multiple first training texts, and to obtain the category label data of each first training text.

[0228] In one possible implementation, the apparatus further includes: a first updating unit;

[0229] The sampling unit is also used to randomly sample multiple first training texts to obtain multiple texts to be annotated; the total number of multiple texts to be annotated is greater than or equal to the second number, and the second number is less than the first number;

[0230] The annotation unit is also used to respond to the second object's category annotation operation on each text to be annotated according to the preset classification business requirements, and to obtain the annotation category data of each text to be annotated;

[0231] The determining unit 1004 is further configured to determine that the category label data of each first training text is correctly labeled if the consistency between the multiple category label data corresponding to the multiple texts to be annotated and the multiple annotation category data corresponding to the multiple texts to be annotated is greater than or equal to a preset consistency level.

[0232] The first update unit is used to update the category label data of each first training text if the consistency level is less than the preset consistency level.

[0233] In one possible implementation, the apparatus further includes: a verification unit and a second update unit;

[0234] The prediction unit 1001 is also used to predict the category of each test text among multiple test texts that are different from multiple first training texts through a preset classification model, and to obtain the third prediction probability of each test text belonging to multiple preset categories.

[0235] The verification unit is used to perform model verification on the preset classification model based on the third prediction probability of each test text to which it belongs, the category label data of each test text, and the preset verification index, and to obtain the model verification result of the preset classification model.

[0236] The second update unit is used to update multiple first training texts based on multiple historical texts to obtain multiple second training texts if the model validation results do not meet the preset validation conditions.

[0237] The second update unit is also used to perform knowledge distillation-based model training on the first classification model and the second classification model based on multiple second training texts and the category label data annotated for each second training text, and update the preset classification model.

[0238] In one possible implementation, the preset verification metrics include any one of confusion matrix, precision, and recall.

[0239] In one possible implementation, clustering unit 1002 is specifically used for:

[0240] Determine multiple initial center points corresponding to multiple preset categories;

[0241] Clustering is performed on multiple texts to be classified based on the distance between each text to be classified and each initial centroid to determine the preset category to which each text to be classified belongs;

[0242] Update each initial centroid based on multiple unclassified texts clustered under the preset category corresponding to each initial centroid;

[0243] Repeat the clustering process based on the distance between each text to be classified and each initial centroid until the clustering converges;

[0244] Based on the preset category to which each text to be classified belongs after clustering convergence, determine the second distribution probability of each text to be classified belonging to multiple preset categories.

[0245] In one possible implementation, when the text classification method is applied to a text classification response scenario, the determining unit 1004 is also used to determine the text to be replied to that matches the classification category of the text to be classified.

[0246] As can be seen from the above technical solution, a pre-defined classification model is obtained by pre-training the first and second classification models using knowledge distillation based on multiple first training texts and the category labels labeled for each first training text. Since the complexity of the first classification model is greater than that of the second classification model, each text to be classified is input into the pre-defined classification model for category prediction, outputting the first distribution probability of each text belonging to multiple pre-defined categories. Because the knowledge distillation-based model training method reduces model complexity while minimizing the reduction in classification accuracy, the pre-defined classification model can, to a certain extent, guarantee the classification performance of the texts to be classified. Furthermore, clustering the multiple texts to be classified yields the second distribution probability of each text belonging to multiple pre-defined categories. This clustering method can compensate for the weakness of the pre-defined classification model caused by a small total number of first training texts or an imbalanced category distribution. For each text to be classified, the first probability distribution and the second probability distribution of the text to be classified belonging to multiple preset categories are fused to obtain the fused probability distribution of the text to be classified belonging to multiple preset categories. This fusion method makes the fused probability distribution more accurate than the first and second probability distributions. Taking the preset category corresponding to the highest probability component in the fused probability distribution as the classification category of the text to be classified can improve the classification accuracy of the text to be classified, thereby improving the classification effect.

[0247] This application also provides a computer device, which may be a server, see [link to previous document]. Figure 11 , Figure 11This is a structural diagram of a server provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance. It may include one or more processors, such as a Central Processing Unit (CPU) 1122, and a memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 can be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the server 1100.

[0248] Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0249] In this embodiment, the central processing unit 1122 in server 1100 can perform the following steps:

[0250] The class of each text to be classified is predicted by a pre-defined classification model, and the first distribution probability of each text to be classified to belong to multiple pre-defined classes is obtained. The pre-defined classification model is obtained by training the first classification model and the second classification model based on knowledge distillation based on multiple first training texts and the class label data of each first training text. The model complexity of the first classification model is greater than that of the second classification model.

[0251] Clustering is performed on multiple texts to be classified to obtain the second distribution probability of each text to be classified belonging to multiple preset categories;

[0252] For each text to be classified, the first distribution probability of the text to be classified belonging to multiple preset categories and the second distribution probability of the text to be classified belonging to multiple preset categories are fused to obtain the fused distribution probability of the text to be classified belonging to multiple preset categories;

[0253] The classification category of the text to be classified is determined based on the preset category corresponding to the highest probability component in the fusion distribution probability.

[0254] The computer device provided in this application embodiment can also be a terminal, see [link to relevant documentation]. Figure 12 , Figure 12 This is a structural diagram of a terminal provided in an embodiment of this application. Taking a smartphone as an example, the smartphone includes components such as a radio frequency (RF) circuit 1210, a memory 1220, an input unit 1230, a display unit 1240, a sensor 1250, an audio circuit 1260, a wireless Fidelity (WiFi) module 1270, a processor 1280, and a power supply 1290. The input unit 1230 may include a touch panel 1231 and other input devices 1232, the display unit 1240 may include a display panel 1241, and the audio circuit 1260 may include a speaker 1261 and a microphone 1262. Those skilled in the art will understand that... Figure 12 The smartphone structure shown does not constitute a limitation on smartphones and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0255] The memory 1220 can be used to store software programs and modules. The processor 1280 executes various functions and data processing of the smartphone by running the software programs and modules stored in the memory 1220. The memory 1220 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the smartphone (such as audio data, phonebook, etc.). In addition, the memory 1220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0256] The processor 1280 is the control center of the smartphone, connecting various parts of the smartphone via various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 1220 and calling data stored in the memory 1220. Optionally, the processor 1280 may include one or more processing units; preferably, the processor 1280 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1280.

[0257] In this embodiment, the processor 1280 in the smartphone can perform the following steps:

[0258] The class of each text to be classified is predicted by a pre-defined classification model, and the first distribution probability of each text to be classified to belong to multiple pre-defined classes is obtained. The pre-defined classification model is obtained by training the first classification model and the second classification model based on knowledge distillation based on multiple first training texts and the class label data of each first training text. The model complexity of the first classification model is greater than that of the second classification model.

[0259] Clustering is performed on multiple texts to be classified to obtain the second distribution probability of each text to be classified belonging to multiple preset categories;

[0260] For each text to be classified, the first distribution probability of the text to be classified belonging to multiple preset categories and the second distribution probability of the text to be classified belonging to multiple preset categories are fused to obtain the fused distribution probability of the text to be classified belonging to multiple preset categories;

[0261] The classification category of the text to be classified is determined based on the preset category corresponding to the highest probability component in the fusion distribution probability.

[0262] According to one aspect of this application, a computer-readable storage medium is provided for storing a computer program for implementing the text classification methods described in the foregoing embodiments.

[0263] According to one aspect of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0264] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0265] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0266] In specific embodiments of this application, if the preset object, actual interaction data and other related data involve user information, when the above embodiments of this application are applied to specific products or technologies, it is necessary to obtain separate permission or consent from the user, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0267] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0268] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0269] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0270] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0271] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A text classification method, characterized in that, The method includes: The category of each text to be classified is predicted by a preset classification model to obtain the first distribution probability of each text to be classified belonging to multiple preset categories. The preset classification model is obtained by training the first classification model and the second classification model based on knowledge distillation based on multiple first training texts and the category label data of each first training text. The model complexity of the first classification model is greater than that of the second classification model. Cluster the multiple texts to be classified to obtain the second distribution probability of each text to be classified belonging to the multiple preset categories; For each text to be classified, the first distribution probability of the text to be classified belonging to the plurality of preset categories and the second distribution probability of the text to be classified belonging to the plurality of preset categories are fused to obtain the fused distribution probability of the text to be classified belonging to the plurality of preset categories; The classification category of the text to be classified is determined based on the preset category corresponding to the highest probability component in the fusion distribution probability.

2. The method according to claim 1, characterized in that, The process of fusing the first distribution probability of the text to be classified to the multiple preset categories and the second distribution probability of the text to be classified to the multiple preset categories to obtain the fused distribution probability of the text to be classified to the multiple preset categories includes: Determine the first weight corresponding to the first probability distribution and the second weight corresponding to the second probability distribution; the sum of the first weight and the second weight is 1; The first distribution probability, the first weight, the second distribution probability, and the second weight of the multiple preset categories to which the text to be classified belong are weighted to obtain the fused distribution probability of the multiple preset categories to which the text to be classified belongs.

3. The method according to claim 1, characterized in that, The training steps of the preset classification model include: For each first training text, the first pre-trained model in the first classification model is used to predict the category of the first training text, and the third distribution probability of the first training text belonging to the multiple preset categories is obtained. By using the normalization layer in the first classification model, the third distribution probability is normalized based on knowledge distillation to obtain the target distribution probability of the multiple preset categories to which the first training text belongs. The second pre-trained model in the second classification model is used to predict the category of each first training text to obtain the fourth distribution probability of the multiple preset categories to which the first training text belongs; the model complexity of the first pre-trained model is greater than that of the second pre-trained model. By using the first normalization layer in the second classification model, the fourth distribution probability is normalized based on knowledge distillation to obtain the first predicted probability of the first training text belonging to the multiple preset categories. The fourth distribution probability is normalized by the second normalization layer in the second classification model to obtain the second predicted probability of the first training text belonging to the multiple preset categories. Based on the first predicted probability, the target distribution probability, the second predicted probability, the category label data of the first training text annotation, and the preset loss function of the second classification model, train the model parameters of the second classification model until the model converges; The second pre-trained model and the second normalization layer in the trained second classification model are determined as the preset classification model.

4. The method according to claim 3, characterized in that, The preset loss function includes a first loss function and a second loss function; the step of training the model parameters of the second classification model until the model converges based on the first predicted probability, the target distribution probability, the second predicted probability, the category label data of the first training text annotation, and the preset loss function of the second classification model includes: Based on the first predicted probability, the target distribution probability, and the first loss function, determine the first loss corresponding to the first training text; Based on the second predicted probability, the category label data, and the second loss function, determine the second loss corresponding to the first training text; The first loss, the third weight corresponding to the first loss, the second loss, and the fourth weight corresponding to the second loss are weighted to obtain the total loss corresponding to the first training text; the sum of the third weight and the fourth weight is 1. The model parameters of the second classification model are trained based on the total loss until the model converges.

5. The method according to claim 1, characterized in that, The steps for obtaining the multiple first training texts are as follows: Randomly sample multiple historical texts collected to obtain multiple first training texts; the total number of the multiple first training texts is greater than or equal to the first number.

6. The method according to claim 5, characterized in that, The steps for obtaining the category label data for each first training text are as follows: In response to the first object performing a category labeling operation on each of the multiple first training texts according to the preset classification business requirements, the category label data of each of the first training texts is obtained.

7. The method according to claim 6, characterized in that, Before training the preset classification model, the method further includes: Randomly sample the plurality of first training texts to obtain a plurality of texts to be annotated; the total number of the plurality of texts to be annotated is greater than or equal to a second number, and the second number is less than the first number. In response to the second object's category labeling operation for each text to be annotated according to the preset classification business requirements, the annotation category data of each text to be annotated is obtained; If the consistency between the multiple category label data corresponding to the multiple texts to be annotated and the multiple annotation category data corresponding to the multiple texts to be annotated is greater than or equal to a preset consistency level, it is determined that the category label data of each first training text is correctly labeled. If the consistency level is less than the preset consistency level, update the category label data of each first training text.

8. The method according to claim 5, characterized in that, Before predicting the category of each text to be classified among multiple texts using a preset classification model, the method further includes: The preset classification model is used to predict the category of each test text among a plurality of test texts that are different from the plurality of first training texts, and a third prediction probability of the plurality of preset categories to which each test text belongs is obtained. Based on the third predicted probability of each test text belonging to multiple preset categories, the category label data of each test text, and preset verification indicators, the preset classification model is validated to obtain the model validation result of the preset classification model. If the model verification result does not meet the preset verification conditions, the multiple first training texts are updated according to the multiple historical texts to obtain multiple second training texts; Based on the plurality of second training texts and the category label data annotated for each second training text, the first classification model and the second classification model are trained using knowledge distillation, and the preset classification model is updated.

9. The method according to claim 8, characterized in that, The preset verification metrics include any one of confusion matrix, precision, and recall.

10. The method according to claim 1, characterized in that, The step of clustering the plurality of texts to be classified to obtain the second distribution probability of each text to be classified belonging to the plurality of preset categories includes: Determine multiple initial center points corresponding to the multiple preset categories; The multiple texts to be classified are clustered based on the distance between each text to be classified and each initial centroid to determine the preset category to which each text to be classified belongs; Each initial centroid is updated based on multiple unclassified texts clustered under a preset category corresponding to each initial centroid. Repeat the process of clustering the multiple texts to be classified based on the distance between each text to be classified and each initial centroid until the clustering converges. Based on the preset category to which each text to be classified belongs after clustering convergence, determine the second distribution probability of the multiple preset categories to which each text to be classified belongs.

11. The method according to any one of claims 1-10, characterized in that, When the text classification method is applied to a text classification response scenario, the method further includes: Based on the category to which the text to be classified belongs, determine the text to be replied to that matches the category.

12. A text classification device, characterized in that, The device includes: a prediction unit, a clustering unit, a fusion unit, and a determination unit; The prediction unit is used to predict the category of each text to be classified among multiple texts to be classified by using a preset classification model, and to obtain the first distribution probability of multiple preset categories to which each text to be classified belongs; the preset classification model is obtained by training a first classification model and a second classification model based on knowledge distillation based on multiple first training texts and the category label data labeled in each first training text, and the model complexity of the first classification model is greater than that of the second classification model. The clustering unit is used to perform clustering processing on the plurality of texts to be classified, and to obtain the second distribution probability of the plurality of preset categories to which each text to be classified belongs; The fusion unit is used to perform fusion processing on the first distribution probability of the multiple preset categories to which the text to be classified belongs and the second distribution probability of the multiple preset categories to which the text to be classified belongs for each text to be classified, so as to obtain the fused distribution probability of the multiple preset categories to which the text to be classified belongs. The determining unit is used to determine the classification category to which the text to be classified belongs based on the preset category corresponding to the maximum probability component in the fusion distribution probability.

13. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to execute the method according to any one of claims 1-11 according to instructions in the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for implementing the method according to any one of claims 1-11.

15. A computer program product, comprising a computer program, characterized in that, When it is run on a computer device, it causes the computer device to perform the method according to any one of claims 1-11.

Images