Text classification model training method and device, text classification method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides a text classification model training method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: clustering an initialized text to obtain a plurality of clusters, and screening out a seed text of each cluster; constructing a first prompt word corresponding to the seed sample of each cluster, and labeling the seed text of each cluster based on the first prompt word to obtain a label of the seed text of each cluster; taking a label of the seed text as a pseudo label of the initialized text; and training a text classification model based on the initialized text carrying the pseudo tag. By means of the text classification method and device, high dependence on manpower and professional knowledge can be relieved in the training process of the text classification model, then the training cost is reduced, and the to-be-classified text can be quickly and accurately classified through the trained text classification model.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a training method for a text classification model, a text classification method, a device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Effectively converting unstructured text information into category labels is extremely critical for subsequent data analysis and application. However, the label classification methods and text-based label classifier technology in related technologies are still highly dependent on professional knowledge in specific fields and manual intervention for detailed sorting and labeling. The entire conversion process is costly and requires a lot of time and resources, making it difficult to adapt to the high requirements for efficiency and accuracy in the big data era. Summary of the invention

[0003] The embodiments of the present application provide a training method, device, electronic device, computer-readable storage medium and computer program product for a text classification model, which can reduce the high dependence on manpower and professional knowledge in the training process of the text classification model, thereby reducing training costs, and can quickly and accurately classify the text to be classified through the trained text classification model.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a method for training a text classification model, including:

[0006] Cluster the initial text to obtain multiple clusters, and filter out the seed text of each cluster;

[0007] Constructing a first prompt word corresponding to the seed text of each cluster, and labeling the seed text of each cluster based on the first prompt word to obtain a label of the seed text of each cluster;

[0008] Using the label of the seed text as the pseudo label of the initialization text;

[0009] A text classification model is trained based on the initialization text carrying the pseudo label.

[0010] The present application also provides a text classification method, including:

[0011] Get the text to be classified;

[0012] The text to be classified is classified by a text classification model to obtain a classification result of the text to be classified.

[0013] The present application provides a training device for a text classification model, including:

[0014] A determination module is used to cluster the initialization text to obtain multiple clusters and filter out the seed text of each cluster;

[0015] A generation module, used to construct a first prompt word corresponding to the seed text of each cluster, and label the seed text of each cluster based on the first prompt word to obtain a label of the seed text of each cluster;

[0016] A processing module, used for using the label of the seed text as a pseudo label of the initialization text;

[0017] A training module is used to train a text classification model based on the initialization text carrying the pseudo label.

[0018] The present application also provides a text classification device, including:

[0019] An acquisition module is used to obtain the text to be classified;

[0020] The classification module is used to classify the text to be classified through a text classification model to obtain a classification result of the text to be classified.

[0021] An embodiment of the present application provides an electronic device, including:

[0022] A memory for storing executable instructions;

[0023] The processor is used to implement the text classification model training method or text classification method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0024] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for implementing a text classification model training method or a text classification method provided in an embodiment of the present application when executed by a processor.

[0025] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions, which are used to implement the text classification model training method or text classification method provided in the embodiment of the present application when executed by a processor.

[0026] The embodiments of the present application have the following beneficial effects:

[0027] On the one hand, by clustering the initialization text to obtain multiple clusters, and screening out the seed text of each cluster, and using the label of the seed text as the pseudo-label of the initialization text, the amount of text data that needs to be annotated can be greatly reduced; on the other hand, by constructing the prompt words corresponding to the seed text of each cluster, and automatically generating the label of the seed text of each cluster based on the prompt words, the high dependence on manpower and professional knowledge in the training process of the text classification model can be reduced, thereby reducing the training cost. In addition, since the labels generated based on the prompt words are high-quality labels, the trained text classification model has a higher accuracy rate. In other words, the trained text classification model can be used to quickly and accurately classify the classified text. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 1 is a schematic diagram of the architecture of a text classification model training system 100 provided in an embodiment of the present application;

[0029] Figure 2A is a schematic diagram of the structure of an electronic device 500 provided in an embodiment of the present application;

[0030] Figure 2B is a schematic diagram of the structure of an electronic device 600 provided in an embodiment of the present application;

[0031] Figure 3 It is a flowchart of a training method for a text classification model provided in an embodiment of the present application;

[0032] Figure 4 It is a flowchart of a training method for a text classification model provided in an embodiment of the present application;

[0033] Figure 5 It is a flowchart of a training method for a text classification model provided in an embodiment of the present application;

[0034] Fig. 6A It is a flowchart of a training method for a text classification model provided in an embodiment of the present application;

[0035] Figure 6B It is a flowchart of a training method for a text classification model provided in an embodiment of the present application;

[0036] Fig. 7A It is a flowchart of a training method for a text classification model provided in an embodiment of the present application;

[0037] Figure 7B It is a flowchart of a training method for a text classification model provided in an embodiment of the present application;

[0038] Figure 8 It is a flowchart of a text classification method provided in an embodiment of the present application;

[0039] Fig. 9 It is a process diagram of the training method of the text classification model provided in the embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0041] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0042] It is understandable that in the embodiments of the present application, related data such as user information (such as initialization text data) is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards.

[0043] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0044] In the following description, the terms "first\second\..." involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\..." can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0046] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0047] 1) Prompt: In the field of artificial intelligence, prompts refer to text instructions used to guide the model on how to generate responses; an effective prompt usually consists of five parts: role definition, instructions, context, examples, and output format.

[0048] 2) Large Language Model (LLM): Large language models are machine learning models designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, etc. Large language models are characterized by their large scale and billions of parameters that help them learn complex patterns in language data. They are usually based on deep learning architectures and can be used to generate natural language text and understand its meaning. Through training, the model can learn the statistical laws and semantic associations of language to build a huge language knowledge base, thereby simulating human language understanding and generation capabilities.

[0049] 3) Seed text: Seed text refers to representative text selected from each cluster category of the initialization text.

[0050] 4) Clustering: Clustering is an unsupervised learning technique that aims to divide a data set into several groups (clusters) so that the data points in the same group are more similar to each other, while the data points in different groups are less similar. Common clustering algorithms include K-means, hierarchical clustering, density clustering (DBSCAN), and Gaussian mixture model.

[0051] 5) Silhouette Coefficient: Silhouette coefficient is an indicator for evaluating clustering effect, and its value range is [-1, 1]. The closer the silhouette coefficient is to 1, the better the clustering effect. The basic idea of ​​calculating the silhouette coefficient is to calculate the similarity (distance) between each text and its own cluster center and the similarity (distance) between each text and other cluster centers, and then calculate the difference between the two similarities.

[0052] Effectively converting unstructured text information into category labels is extremely critical for subsequent data analysis and application; however, the label classification methods used in related technologies and the construction of text-based label classifier technologies are still highly dependent on professional knowledge in specific fields and manual intervention for detailed sorting and labeling, which leads to high costs for the entire conversion process and requires a lot of time and resources, making it difficult to adapt to the high requirements for efficiency and accuracy in the big data era. Therefore, developing more automated and intelligent text classification technologies to reduce excessive reliance on manpower and professional knowledge and to quickly and accurately convert unstructured text into structured category labels has become an important issue that needs to be solved.

[0053] Based on this, the embodiments of the present application provide a training method for a text classification model, a text classification method, a device, an electronic device, a computer-readable storage medium, and a computer program product, which can reduce the high dependence on manpower and professional knowledge in the training process of the text classification model, thereby reducing the training cost, and can quickly and accurately classify the classified text through the trained text classification model. The electronic device provided in the embodiments of the present application can be implemented as a server, or implemented by a server and a terminal in collaboration. The following is an example of an example in which a server and a terminal collaborate to implement the training method of the text classification model provided in the embodiments of the present application.

[0054] For example, see Figure 1 , Figure 1 1 is a schematic diagram of the architecture of a text classification model training system 100 provided in an embodiment of the present application, in order to implement a training application supporting a text classification model, such as Figure 1 As shown, the training system 100 of the text classification model includes: a server 200, a network 300, and a terminal 400. The terminal 400 is connected to the server 200 via the network 300, wherein the network 300 can be a local area network or a wide area network, or a combination of the two; the terminal 400 is a terminal associated with the user, and a client 410 runs on the terminal 400. The client 410 can be various types of clients, such as a dedicated model training client, a text annotation client, and a browser.

[0055] In some embodiments, the terminal 400 is used to input the collected initialization text in the client 410, and the terminal 400 sends the initialization text to the server 200 through the network 300. The server 200 clusters the initialization text to obtain multiple clusters, and screens out the seed text of each cluster; then, the server 200 can construct a first prompt word corresponding to the seed text of each cluster, and label the seed text of each cluster based on the first prompt word to obtain the label of the seed text of each cluster; then, the server 200 uses the label of the seed text as a pseudo-label of the initialization text; then, the server 200 trains the text classification model based on the initialization text carrying the pseudo-label, thereby obtaining the trained text classification model; finally, the server 200 can send the trained text classification model to the terminal 400 through the network 300, that is, the trained text classification model can be automatically deployed on the client 410 of the terminal 400.

[0056] It should be noted that the technical solution provided by the embodiments of the present application can be applied to a variety of application scenarios, such as text classification, sentiment analysis, machine learning training data annotation, etc. For example, the trained text classification model can be applied to the text classification scenario, and the trained text classification model can be used to quickly and accurately classify the classified text; for example, the trained text classification model can be applied to the machine learning training data annotation scenario. Assuming that the training data is text data, the trained text classification model can be used to automatically and quickly annotate the training data, and a high-quality training data set can be generated through automatic annotation.

[0057] In other embodiments, the embodiments of the present application can also be implemented with the aid of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data calculation, storage, processing, and sharing.

[0058] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources.

[0059] For example, Figure 1The server 200 in the example may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal 400 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, etc., but is not limited thereto. The terminal 400 and the server 200 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0060] The following is a description of the structure of an electronic device for implementing a training method for a text classification model provided in an embodiment of the present application. Figure 2A , Figure 2A 2 is a schematic diagram of the structure of an electronic device 500 provided in an embodiment of the present application. The electronic device 500 shown in FIG2 includes: at least one processor 510, a memory 550 and at least one network interface 520. The various components in the electronic device 500 are coupled together through a bus system 530. It can be understood that the bus system 530 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 530 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 2A Various buses are labeled as bus system 530 .

[0061] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0062] The memory 540 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 540 may optionally include one or more storage devices that are physically remote from the processor 510.

[0063] The memory 540 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 540 described in the embodiments of the present application is intended to include any suitable type of memory.

[0064] In some embodiments, memory 540 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0065] Operating system 541, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0066] A network communication module 542, for reaching other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 include: Bluetooth, Wireless Authentication (WiFi), and Universal Serial Bus (USB);

[0067] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2A A training device 543 of a text classification model stored in a memory 540 is shown, which may be software in the form of a program and a plug-in, etc., and includes the following software modules: a determination module 5431, a generation module 5432, a processing module 5433 and a training module 5434. These modules are logical and can therefore be arbitrarily combined or further split according to the functions implemented.

[0068] The structure of the electronic device for implementing the text classification method provided in the embodiment of the present application is further described below. Taking the electronic device 600 as a server as an example, see Figure 2B , Figure 2B is a schematic diagram of the structure of an electronic device 600 provided in an embodiment of the present application, Figure 2B The electronic device 600 shown includes a text classification device 643, which can be software in the form of a program and a plug-in, etc., including the following software modules: an acquisition module 6431 and a classification module 6432. The functions of each module will be explained below.

[0069] It should be noted that Figure 2BThe electronic device 600 shown includes a processor 610, a network interface 620, a bus system 630, a memory 640, an operating system 641, and a network communication module 642. Figure 2A The corresponding modules included in have the same structure and the same function, and the embodiments of the present application will not be repeated here.

[0070] The training method of the text classification model provided in the embodiment of the present application will be specifically described below in combination with the exemplary application and implementation of the terminal provided in the embodiment of the present application.

[0071] See also Figure 3 , Figure 3 is a flowchart of a training method for a text classification model provided in an embodiment of the present application, which will be combined with Figure 3 The steps shown are explained.

[0072] In step 101, the initialization text is clustered to obtain a plurality of clusters.

[0073] Here, the initialization text refers to the text without pseudo labels.

[0074] It should be noted that the initialization text data mainly includes relevant texts in the field to be analyzed. For example, for user emotion tasks in telemarketing scenarios, the initialization text can be the existing text data of conversations between telemarketing agents and users. In addition, there are many ways to obtain the initialization text data, including manual writing, web crawler capture, open data sets, database queries and other means. The specific acquisition method needs to be determined based on actual conditions and is not specifically limited here.

[0075] In some embodiments, see Figure 4 , Figure 4 is a flowchart of a method for training a text classification model provided in an embodiment of the present application, such as Figure 4 As shown, Figure 3 Step 101 shown may be performed by Figure 4 Steps 1011 to 1013 shown are implemented by combining Figure 4 The steps shown are explained.

[0076] In step 1011, the initialization text is embedded to obtain a text vector of the initialization text.

[0077] In some embodiments, first, the initialization text is preprocessed (including word segmentation, stop word removal, stem extraction, standardization and other operations); then, a suitable word embedding model is selected (such as Word2Vec, BERT and other models); then, the selected word embedding model is used for model training to obtain a trained model; finally, the preprocessed initialization text is input into the word embedding model, and the text vector of the initialization text is output.

[0078] For example, suppose there is an initialization text: "Climate change is a global problem that requires international cooperation to solve". First, the initialization text is preprocessed to obtain the preprocessed initialization text: "climate change", "global", "problem", "international", "cooperation", "solution"; then, the Word2Vec model is used to train the text data to obtain the trained model; finally, the preprocessed initialization text is input into the trained model, and the model converts the processed initialization text into the corresponding embedding vector, and the output text vector of the initialization text can be expressed as [0.12, -0.34, 0.56, -0.23, 0.45, -0.12, 0.78, -0.45, 0.23].

[0079] In step 1012, the text vectors are clustered to obtain an initial clustering result.

[0080] In some embodiments, first, a suitable clustering algorithm (such as K-means, hierarchical clustering, density clustering, etc.) is selected, and suitable clustering parameters are set according to user needs (for example, the user needs to obtain 3 clusters); then, the text vector is clustered using the selected clustering algorithm to obtain an initial clustering result.

[0081] Continuing with the above example, assume that the text vector of the initialization text is [0.12, -0.34, 0.56, -0.23, 0.45, -0.12, 0.78, -0.45, 0.23]. Next, select the K-means clustering algorithm to cluster the text vector, and set the number of clusters to 3 according to user requirements. Subsequently, use the K-means clustering algorithm to cluster the text vector and obtain preliminary clustering results, which preliminarily divide the text vector into 3 clusters.

[0082] In step 1013, the initial clustering result is adjusted based on the silhouette coefficient to obtain multiple clusters.

[0083] Here, each cluster includes a portion of initialization text.

[0084] In some embodiments, first, for each initialization text, the similarity distance between the current initialization text and its own cluster center, as well as the similarity distance between the current initialization text and each other cluster center are calculated, and the difference between the two similarity distances is calculated to obtain the silhouette coefficient of the current initialization text; then, based on the silhouette coefficient, the initialization text with a lower silhouette coefficient is identified, and the identified text is divided into other clusters; then, the clustering model is retrained using the adjusted clusters, and the above steps are repeated until the calculated silhouette coefficient is no longer significantly improved or a predetermined number of iterations is reached; finally, multiple clusters after clustering processing are obtained.

[0085] For example, suppose there is a text dataset containing 20 news articles. Using the K-means algorithm for initial clustering, these 20 articles are divided into 3 clusters. The initial clustering results are: Cluster 1 includes news A, news B, news C, news D; Cluster 2 includes news E, news F, news G, news H, news I; Cluster 3 includes news J, news K, news L, news M, news N, news O, news P. Then, the silhouette coefficient of each text is calculated, and the silhouette coefficients are: Cluster 1 is 0.12, 0.23, 0.34, 0.45; Cluster 2 is 0.56, 0.67, 0.78, 0.89, 0.90; Cluster 3 is 0.23, 0.34, 0.45, 0.56, 0.67, 0.78. According to the calculation results, it can be seen that the silhouette coefficient of cluster 2 is generally high, close to cluster 1, indicating that the similarity within cluster 2 is very high and the similarity with other clusters is low, and the clustering quality is good; however, the silhouette coefficient of cluster 1 is low, indicating that there are some texts in cluster 1 that should be assigned to other clusters.

[0086] Next, the initial clustering results are adjusted according to the silhouette coefficient. News A, news B, and news C in cluster 1 are moved to cluster 2 because their silhouette coefficients are lower, and the silhouette coefficient of cluster 2 is higher, indicating that they may be more suitable for cluster 2; news D in cluster 1 is moved to cluster 3 because its silhouette coefficient is the lowest, and the silhouette coefficients of other texts in cluster 3 are also low, indicating that they may be semantically closer; then, after adjustment, new clustering results are obtained, that is, the three clusters are: Cluster 1 includes news D, news L, news M, news N, news O, and news P; Cluster 2 includes news A, news B, news C, news E, news F, news G, news H, and news I; Cluster 3 includes news J and news K.

[0087] In step 102, the seed text of each cluster is screened out.

[0088] Here, seed text refers to representative text that can represent the theme or characteristics of the entire cluster and can help understand the theme and content of each cluster.

[0089] In some embodiments, see Figure 5 , Figure 5 is a flowchart of a method for training a text classification model provided in an embodiment of the present application, such as Figure 5 As shown, Figure 3 Each cluster in step 102 may perform Figure 5 Steps 1021 to 1022 shown in FIG. Figure 5 The steps shown are explained.

[0090] In step 1021 , the centers of the plurality of text vectors included in the cluster are determined.

[0091] In some embodiments, when each cluster contains multiple text vectors, the center of the text vectors included in the current cluster can be determined by calculating the mean of all text vectors in the current cluster; when there are not enough text vectors in the cluster (for example, the number of text vectors is less than 2), the center of the text vectors included in the current cluster can be manually set.

[0092] For example, suppose cluster 1 includes news A: [0.12, -0.34, 0.56, -0.23, 0.45], news B: [0.45, -0.12, 0.78, -0.45, 0.23], and news C: [0.12, -0.34, 0.56, -0.23, 0.45], then the center of the text vector included in cluster 1 is (news A + news B + news C) / 3, that is, [0.12, -0.36, 0.6, -0.33, 0.49].

[0093] In step 1022, the initialization text corresponding to the text vector whose distance from the center is less than the distance threshold is used as the seed text selected from the cluster.

[0094] In some embodiments, the above-mentioned use of the initialization text corresponding to the text vector whose distance from the center is less than the distance threshold as the seed text screened out from the cluster can be achieved in the following way: randomly extracting N initialization texts from the initialization text corresponding to the text vector whose distance from the center is less than the distance threshold as the seed text screened out from the cluster.

[0095] Here, N is a positive integer greater than or equal to 1.

[0096] In some embodiments, for each cluster, the distance between all text vectors in the current cluster and the center of the current cluster is calculated; then, a distance threshold is determined, where the distance threshold can be set manually as needed, or the system can automatically determine it according to scene conditions, and the distance thresholds between different clusters can be the same or different, which is not specifically limited here; then, for each cluster, the initialization texts with a distance less than the distance threshold are screened out, and N initialization texts are randomly selected from the screened texts as seed texts screened out from the current cluster. In this way, the diversity of the selected seed samples can be guaranteed.

[0097] For example, suppose that the K-means algorithm is used to cluster the word embedding vectors of news articles, and three clusters are obtained. The centers of each cluster are cluster 1 center: [0.12, -0.34, 0.56, -0.23, 0.45], cluster 2 center: [0.45, -0.12, 0.78, -0.45, 0.23], cluster 3 center: [0.78, -0.45, 0.23, 0.12, -0.34]; then, after calculation, it is determined that the distances between each text in each cluster and the cluster center are cluster 1: 0.8, 0.4, 1.2, 6.7, cluster 2: 5.6, 4.7, 3.1, cluster 3: 8.4, 0.2, 0.7, 1.9; then determine The distance threshold of cluster 1 is 1.0, then the texts selected from cluster 1 are the texts corresponding to distance values ​​0.8 and 0.4. The distance threshold of cluster 2 is determined to be 4.0, then the texts selected from cluster 2 are the texts corresponding to distance values ​​3.1. The distance threshold of cluster 3 is determined to be 2.0, then the texts selected from cluster 3 are the texts corresponding to distance values ​​0.2, 0.7 and 1.9. Finally, a text is randomly selected from each cluster as the seed text selected from the cluster. For example, after random selection, the text corresponding to the distance value of 0.8 in cluster 1 is obtained as the seed text of cluster 1, the text corresponding to the distance value of 3.1 in cluster 2 is obtained as the seed text of cluster 2, and the text corresponding to the distance value of 0.2 in cluster 3 is obtained as the seed text of cluster 3.

[0098] In step 103, the first prompt word corresponding to the seed text of each cluster is constructed.

[0099] In some embodiments, first, determine the type of text (such as news reports, stories, conversations, etc.) and style (such as formal, humorous, poetic, etc.) generated by the model; then, select some high-quality reference texts, which should match the target type and style. The reference text here can be a ready-made text or an example manually written according to the target; then, design a first prompt word containing the reference text and a blank area, and the blank area is the part that the model needs to fill in; then, provide some guidance information in the first prompt word, such as the length of the text, language style, keywords or themes, etc., to help the model better understand the task requirements; finally, before the model enters the first prompt word, preview the effect of the first prompt word to ensure that it can effectively guide the model to generate the required type of text, and can make corresponding adjustments to the first prompt word based on the preview result.

[0100] For example, suppose the model is required to generate a news report about environmental protection. First, some reference text is needed, such as the beginning and end of a news report about environmental protection. Next, a prompt word can be designed, which contains the beginning and end of the reference text, as well as a blank area, so that the model can fill in the news content in the middle. The prompt word can be expressed as "News report beginning: Recently, the problem of global climate change has become increasingly serious, and governments around the world have taken action to alleviate this situation. As part of the earth, we should also do our part to protect our environment. [Main part of the news report] Ending: In short, protecting the environment is everyone's responsibility. Let us work together to create a better future for our earth." In this way, the model can better understand the task requirements and generate text that meets the task requirements to meet user needs.

[0101] In step 104, the seed text of each cluster is labeled based on the first prompt word to obtain a label of the seed text of each cluster.

[0102] In some embodiments, see Fig. 6A , Fig. 6A is a flowchart of a method for training a text classification model provided in an embodiment of the present application, such as Fig. 6A As shown, Figure 3 Step 104 shown may be performed by Fig. 6A Steps 1041A to 1042A shown in the figure are implemented by combining Fig. 6A The steps shown are explained.

[0103] In step 1041A, the first prompt word and the seed text of each cluster are filled into a preset template to obtain second prompt information.

[0104] Here, the first hint word is used to indicate the label of the seed text for generating each cluster.

[0105] In some embodiments, first, a preset template needs to be selected. Here, the preset template can clearly know that the large language model generates a specific type of text. If no suitable preset template is found, a template can be customized according to needs; then, the preset template structure is analyzed to determine the position in the template where the first prompt word and the seed text of each cluster need to be filled, so as to ensure that the template can accommodate the first prompt word and the seed text of each cluster, and can be correctly understood and used by the template; then, a suitable position is found in the template, and the first prompt word and the seed text of each cluster are filled into the corresponding positions respectively, so as to obtain the second prompt information, so that the large language model can generate processing based on the second prompt information.

[0106] For example, suppose in a news report scenario, "environmental protection" is determined as the first prompt word, and a news report on environmental protection is selected as the seed text. A news report template is selected according to the needs, which contains the basic structure of the news report, such as the title, introduction, body and ending; then, "environmental protection" is filled into the title part of the template, and the seed text is filled into the body part of the template. The template is adjusted to ensure that the format and position of the first prompt word and the seed text are appropriate; finally, the template filled with the first prompt word and the seed text is used as the second prompt information, and it is used to generate a label about the news report through a large language model.

[0107] In step 1042A, based on the second prompt information, a large language model is used to generate a label for the seed text of each cluster.

[0108] In some embodiments, after obtaining the second prompt information, the second prompt information is used as input and generated by a large language model. The large language model first generates new text content based on the prompt information. Then, the large language model analyzes the new text content (for example, calculating text similarity, evaluating text quality, manual review, etc.); then, based on the analysis results, a label is assigned to the seed text of each cluster. The label can be a description of the theme, style, emotion, etc. of the text.

[0109] Continuing with the above example, after obtaining the second prompt information, the generated second prompt information is used as input and generated and processed by the large language model to obtain a new news report; then, the large language model calculates the similarity difference between the generated new news report and the original text. When the similarity difference is less than the difference threshold, it is determined that a new news report is obtained, and a label for the seed text is generated according to the content of the new news report, such as environmental protection.

[0110] In some embodiments, the above-mentioned generation processing based on the second prompt information through a large language model to obtain a label of the seed text of each cluster can be achieved in the following way: summarizing the text content of the seed text of each cluster based on the second prompt information to obtain a first summary content; comparing the first summary content with the text content of the seed sample of each cluster to determine the problems existing in the first summary content; summarizing the text content of the seed text of each cluster again based on the problems existing in the first summary content to obtain a second summary content; and generating processing based on the second summary content through a large language model to obtain a label of the seed text of each cluster.

[0111] In some embodiments, when the number of seed texts in each cluster is multiple, see Figure 6B , Figure 6B is a flowchart of a method for training a text classification model provided in an embodiment of the present application, such as Figure 6B As shown, Figure 3 Step 104 shown may be performed by Figure 6B Steps 1041B to 1043B shown in the figure are implemented by combining Figure 6B The steps shown are explained.

[0112] In step 1041B, multiple seed texts of each cluster are concatenated to obtain a concatenated text.

[0113] In some embodiments, first, each seed text can be preprocessed, such as removing irrelevant punctuation marks, converting to a text in a unified format, etc., to ensure that the seed text maintains consistency in format and content before splicing; then, the preprocessed seed texts are spliced ​​in a certain order. When splicing, you can consider using some conjunctions or transition sentences to smoothly connect different seed samples, so that the spliced ​​text is more semantically coherent; finally, to ensure that the content of the spliced ​​text is logically consistent, the spliced ​​text needs to be appropriately adjusted and modified to make it more in line with the expected theme or style.

[0114] For example, assume that there are three seed texts in the current cluster, seed text 1 is "Environmental pollution is an increasingly serious problem, and we must take measures to improve our air quality", seed text 2 is "Protecting water resources is the responsibility of each of us, and we must take action to reduce water pollution", and seed text 3 is "Biodiversity is an important asset of our earth, and we must take measures to protect it"; then, the three seed texts are preprocessed as necessary and spliced ​​in a logical order to obtain the spliced ​​text "Environmental pollution is an increasingly serious problem, and we must take measures to improve our air quality. Protecting water resources is the responsibility of each of us, and we must take action to reduce water pollution. Biodiversity is an important asset of our earth, and we must take measures to protect it. Only by working together can we ensure that the environment of our earth is protected and create a better future for our descendants. "In this way, multiple related seed texts are spliced ​​into a coherent text to facilitate subsequent text analysis and generate corresponding text labels.

[0115] In step 1042B, based on the first prompt word and the concatenated text, a large language model is used to generate a label for the concatenated text.

[0116] It should be noted that the implementation of step 1042B in the embodiment of the present application is similar to the implementation of the above-mentioned step 1041A. For details, please refer to the implementation of the above-mentioned step 1041A, which will not be repeated here.

[0117] In step 1043B, the label of the concatenated text is used as the label of the multiple seed texts of each cluster.

[0118] It should be noted that the implementation of step 1043B in the embodiment of the present application is similar to the implementation of the above-mentioned step 1042A. For details, please refer to the implementation of the above-mentioned step 1042A, which will not be repeated here.

[0119] In step 105, the label of the seed text is used as a pseudo label of the initialization text.

[0120] It should be noted that the pseudo-label technology is a semi-supervised learning method, which predicts the labels of the seed samples in each cluster and assigns the labels of the generated seed texts as pseudo-labels to the unlabeled initialization texts in the current cluster for subsequent training of the text classification model. Of course, in addition to the pseudo-labels, the initialization texts can also carry other types of labels, which are not specifically limited in the embodiments of the present application.

[0121] In some embodiments, the above-mentioned use of the label of the seed text as the pseudo label of the initialization text can be implemented in the following manner: performing the following processing for each cluster: using the label of the seed text corresponding to the cluster as the pseudo label of the partial initialization text included in the cluster.

[0122] For example, assume that the K-means algorithm is used to cluster news articles, and the clustering results are divided into three clusters. A text is selected from each cluster as the seed text, and the label obtained after the large language model generates the label, where the label of seed text 1 is "environmental protection", the label of seed text 2 is "economic news", and the label of seed text 3 is "health news"; then according to the cluster of seed text, the pseudo label of the initialization text included in the cluster is determined. Seed text 1 corresponds to cluster 1, and the pseudo label of the initialization text included in cluster 1 is "environmental protection", seed text 2 corresponds to cluster 2, and the pseudo label of the initialization text included in cluster 2 is "economic news", seed text 3 corresponds to cluster 3, and the pseudo label of the initialization text included in cluster 3 is "health news". In this way, the pseudo-label technology can use a large amount of unlabeled text data to train the text classification model, which can effectively improve the generalization ability of the model, and the text classification model can continuously expand its knowledge base during the training process, thereby improving the overall prediction performance of the model.

[0123] In step 106, the text classification model is trained based on the initialization text carrying the pseudo-label.

[0124] In some embodiments, see Fig. 7A , Fig. 7A is a flowchart of a method for training a text classification model provided in an embodiment of the present application, such as Fig. 7A As shown, Figure 3 Step 106 shown may be performed by Fig. 7A Steps 1061 to 1066 shown are implemented by combining Fig. 7A The steps shown are explained.

[0125] In step 1061, the initialized texts carrying pseudo labels are divided into a training set and a test set.

[0126] In some embodiments, first, the initialization texts carrying pseudo labels are combined together to construct a training data set; then, the training data set is divided according to a certain ratio (for example, 7:3) to obtain a training set and a test set.

[0127] For example, assuming that the initialization text is entertainment-type text, the initialization texts with pseudo labels are combined together to form a training data set of entertainment articles. Then, the first 80% of the training data can be used as the training set, and the remaining 20% ​​of the data can be used as the test set. In this way, the training set can be used to train the text classification model, and the test set can be used to evaluate the generalization ability of the model, thereby comprehensively evaluating the performance of the model.

[0128] In step 1062, the text classification model is trained based on the training set, and the accuracy of the trained text classification model is tested based on the test set.

[0129] In some embodiments, after the training data set is divided into data sets, the text data in the training set can be preprocessed, including text cleaning, word segmentation, stop word removal, part-of-speech tagging, stem extraction, word embedding and other steps to reduce noise and enhance the generalization ability of the model, thereby improving the efficiency and quality of model training; then, select a suitable feature extraction method (such as bag-of-words model, TF-IDF, word embedding, etc.) to convert the text into a numerical representation that the model can understand; then, select a suitable text classification model (such as naive Bayes, logistic regression, support vector machine (SVM), recurrent neural network (RNN), convolutional neural network (CNN), long short-term memory network (LSTM), Transformer, etc.) according to the requirements of the task and the characteristics of the data; finally, use the training set data to train the selected model so that the model learns how to predict the category of the text based on the features, thereby obtaining a trained text classification model.

[0130] For example, suppose there is an entertainment article dataset that contains multiple categories of entertainment information, and the dataset is divided into training set, validation set and test set. Then, the training set data is preprocessed, and then TF-IDF is used as the feature extraction method to extract features and obtain the corresponding feature vector. Subsequently, a logistic regression model is selected and the model is trained using the training set data to obtain a trained text classification model.

[0131] In some embodiments, after training the text classification model based on the training set to obtain the trained model, the trained model can be used to predict the test set to obtain a prediction result, wherein the prediction result includes a predicted category label and a corresponding probability score; then, various evaluation indicators are calculated based on the prediction result and the pseudo-label of the test set, such as accuracy, recall rate, F1 score, and confusion matrix; finally, the evaluation indicators are analyzed to obtain the corresponding test results.

[0132] In some embodiments, the initialized text carrying pseudo-labels can also be divided into a training set, a validation set, and a test set; the text classification model is trained based on the training set; the trained text classification model is adjusted based on the validation set; the adjusted text classification model is tested based on the test set to obtain test results.

[0133] It should be noted that the above-mentioned process of training and testing the text classification model based on the training set and the test set is similar to the execution process of step 1061 and step 1062. For details, please refer to the execution process of step 1061 and step 1062, which will not be elaborated here.

[0134] In some embodiments, after obtaining the validation set data, the validation set data is used to evaluate the trained text classification model and various evaluation indicators (such as accuracy, recall rate, F1 score, confusion matrix, etc.) are calculated; then, according to the evaluation indicators, the hyperparameters of the model are adjusted, where the hyperparameters include learning rate, regularization strength, hidden layer size, etc. The adjustment of hyperparameters can be achieved through grid search, random search, Bayesian optimization and other methods; finally, when the performance of the model on the validation set reaches the best, the adjusted best model is saved for easy reproduction in subsequent use.

[0135] For example, suppose a logistic regression model is used to classify a news article dataset. During the training process, the model performance is evaluated through cross-validation and validation set. According to the evaluation results of the validation set, it is found that the recall rate of the model is low in some categories. The model hyperparameters, such as learning rate and regularization strength, are adjusted to improve the recall rate. After the adjustment, the model is retrained and evaluated again using the validation set until the model performance reaches the best or meets the preset conditions, and the adjusted model is saved.

[0136] In step 1063, when the test result indicates that the accuracy of the trained text classification model is less than the accuracy threshold, the incorrectly labeled text in the test set is obtained.

[0137] Here, accuracy refers to the ratio of the number of initialized texts in the test set whose classification results are consistent with the pseudo-labels to the number of all initialized texts included in the test set. The classification results are obtained by the trained text classification model classifying the initialized texts in the test set. The mislabeled texts are initialized texts whose classification results output by the trained text classification model are inconsistent with the pseudo-labels they carry.

[0138] It should be noted that the incorrectly labeled text can be the initialization text in which all prediction result labels are inconsistent with the pseudo-labels, or it can be the initialization text in which only the prediction result labels in the test set are inconsistent with the pseudo-labels, and no specific limitation is made here.

[0139] For example, suppose that the test set contains 10 initialization texts with pseudo-labels. The trained text classification model classifies the 10 initialization texts in the test set to obtain the classification results of the model for each initialization text. Suppose that the classification results of 8 initialization texts in the test set are consistent with the pseudo-labels carried by the initialization texts, and the classification results of 2 initialization texts are inconsistent with the pseudo-labels carried by the initialization texts, that is, 2 incorrectly labeled texts. The accuracy of the trained text classification model is 80%. Assuming the accuracy threshold is 85%, the 2 incorrectly labeled texts in the test set are obtained for subsequent processing.

[0140] In step 1064, a second prompt word is constructed based on the incorrectly annotated text.

[0141] Here, the constructed second prompt word may be the same as or different from the first prompt word, and the specific need is determined according to the model requirements.

[0142] It should be noted that the implementation of step 1063 in the embodiment of the present application is similar to the implementation of the above-mentioned step 103. For details, please refer to the implementation of the above-mentioned step 103, which will not be repeated here.

[0143] In step 1065, the initialization text is labeled based on the second prompt word to obtain a pseudo label regenerated for the initialization text.

[0144] Here, all the initialization texts can be labeled based on the second prompt word to obtain pseudo labels regenerated for all the initialization texts; the incorrectly labeled texts can also be labeled based on the second prompt word to obtain pseudo labels regenerated for the incorrectly labeled texts. The specific method can be determined according to the needs of the model.

[0145] In some embodiments, the above step 1065 can be implemented in the following manner: labeling all the initialization texts based on the second prompt word to obtain pseudo labels regenerated for all the initialization texts. In this way, by regenerating the pseudo labels of all the initialization texts and obtaining pseudo labels regenerated for all the initialization texts, the consistency of the pseudo labels of the initialization texts in the entire data set can be ensured, and higher quality training data can be generated, which helps to make more full use of the training data during model training, thereby improving the prediction accuracy of the model.

[0146] In some embodiments, the above step 1065 can also be implemented in the following manner: based on the second prompt word, only the incorrectly labeled text is labeled to obtain a pseudo-label regenerated for the incorrectly labeled text. In this way, only the pseudo-label of the incorrectly labeled text is regenerated, which can effectively reduce the waste of resources and reduce the training cost of the model; in addition, only the pseudo-label of the incorrectly labeled text is regenerated to ensure that the pseudo-label of the original consistent initialization text remains consistent, avoiding errors caused by unnecessary modifications.

[0147] In some embodiments, see Figure 7B , Figure 7B is a flowchart of a method for training a text classification model provided in an embodiment of the present application, such as Figure 7B As shown, Fig. 7A Step 1065 can be shown by Figure 7B Steps 10651 to 10652 shown in the figure are implemented by combining Figure 7B The steps shown are explained.

[0148] In step 10651, the second prompt word and the initialization text are filled into the preset template to obtain the first prompt information.

[0149] Here, the second prompt word is used to indicate the reason for the inconsistent analysis and to regenerate the pseudo label of the initialization text. The first prompt information and the second prompt information may be the same or different, and are not specifically limited here.

[0150] It should be noted that the implementation of step 10651 in the embodiment of the present application is similar to the implementation of the above-mentioned step 1041A. For details, please refer to the implementation of the above-mentioned step 1041A, which will not be repeated here.

[0151] In step 10652, based on the first prompt information, a generation process is performed through a large language model to obtain a pseudo label regenerated for the initialization text.

[0152] It should be noted that the implementation of step 10652 in the embodiment of the present application is similar to the implementation of the above-mentioned step 1042A. For details, please refer to the implementation of the above-mentioned step 1042A, which will not be repeated here.

[0153] In some embodiments, the above-mentioned generation and processing based on the first prompt information through a large language model to obtain a pseudo-label regenerated for the initialization text can be achieved in the following way: based on the first prompt information, the pseudo-label of the incorrectly annotated text and the prediction result of the incorrectly annotated text by the text classification model are analyzed to determine the reason for the inconsistency between the prediction result and the pseudo-label; based on the reason for the inconsistency, the label generation logic is adjusted, and based on the adjusted label generation logic, the generation and processing is performed through a large language model to obtain a pseudo-label regenerated for the initialization text.

[0154] In step 1066, the text classification model is trained based on the initialization text carrying the regenerated pseudo-labels.

[0155] It should be noted that the implementation method of training the text classification model in step 1066 in the embodiment of the present application is similar to the implementation method of the above-mentioned steps 1061 to 1062. For details, please refer to the implementation methods of the above-mentioned steps 1061 and 1062, which will not be repeated here.

[0156] The text classification method provided in the embodiment of the present application is further described below.

[0157] See also Figure 8 , Figure 8 is a flowchart of the text classification method provided in the embodiment of the present application, which will be combined with Figure 8 The steps shown are explained.

[0158] In step 201, the text to be classified is obtained.

[0159] In some embodiments, first, the goal of the text classification task is clarified (such as sentiment analysis, topic classification, spam detection, etc.); then, according to the goal of the classification task, a suitable text data source is selected (such as social media, news articles, forum posts, emails, etc.); then, text data is obtained from the data source through methods such as data crawlers, API interfaces, manual collection, etc.; finally, the obtained text data is preprocessed (such as cleaning, deduplication, denoising, etc.) to improve the prediction performance and prediction accuracy of the text classification model.

[0160] For example, suppose a sentiment analysis task is required, and the goal is to identify whether the content posted by social media users contains positive or negative emotions. In order to obtain the text to be classified, it is first necessary to determine that the goal of the classification task is sentiment analysis; then, select the social media platform as the data source; then, collect the content posted by users on social media through a data crawler or API interface; then, preprocess the collected data, such as removing irrelevant symbols, filtering stop words, extracting stems, etc.; finally, organize the processed text data into a list or data frame to facilitate the subsequent text classification model to classify the text data.

[0161] In step 202, the text to be classified is classified by a text classification model to obtain a classification result of the text to be classified.

[0162] Here, the text classification model is based on Figure 3 The text classification model is trained using the training method shown.

[0163] In some embodiments, the text to be classified is input into a trained text classification model, which predicts the category of the text based on its features and outputs the predicted category of the text to be classified and the corresponding probability score for further analysis and decision making.

[0164] For example, the news article to be classified is input into the trained text classification model, and the text classification model predicts its category (such as sports, finance, entertainment, etc.) according to the characteristics of the text (such as TF-IDF features); then, the classification prediction result of the text classification model is obtained, including the predicted category and probability score (such as sports 87%, finance 1%, entertainment 12%), then the predicted label of the current news article can be determined as "sports" according to the classification prediction result of the text classification model.

[0165] Next, an exemplary application of the embodiment of the present application in a practical application scenario will be described. The exemplary application describes a training method for a text classification model and a specific implementation process of the text classification method in a data annotation system.

[0166] In some embodiments, see Fig. 9 , Fig. 9 is a process diagram of the training method of the text classification model provided in the embodiment of the present application, which will be combined with Fig. 9 The specific implementation process of the text classification model training method in the data annotation system is explained.

[0167] Phase 1: Get the initialization text.

[0168] Here, relevant texts in the field to be analyzed are collected. For user emotion tasks in telemarketing scenarios, conversations between existing telemarketing agents and users are collected, and user texts are extracted as collected text data.

[0169] Stage 2: Text initialization clustering.

[0170] Here, the collected text data is clustered, and the silhouette coefficient value is used to iteratively obtain the optimal number of clusters. By utilizing the user text collected in stage 1, the user text is embedded using a large language model or other model, and the K-means clustering algorithm is used to cluster the embedded representation of the text, and texts with similar semantics are clustered together. The silhouette coefficient is used to evaluate the performance of the current number of clusters.

[0171] Phase 3: Seed text screening.

[0172] Here, the clustering results obtained in stage 2 are used to obtain the distance between each text and the cluster center, and the 10 data closest to the cluster center are selected to add to the seed text. For example, the seed text can be added by random sampling, which can ensure the diversity of the seed text.

[0173] Stage 4: Construct prompt words and input the seed text into the large language model to obtain the label and definition of the sample text.

[0174] Here, the seed text constructed in stage 3 is used and input into the large language model together with the prompt word to generate the label definition and label name of the sample text.

[0175] The prompt word can be expressed in the form of: "You are a knowledge expert who is good at data analysis, and I am good at text classification tasks. Please generate labels and label definitions for the following text."

[0176] The large language model can generate labels and label definitions for sample text using the following steps:

[0177] (1) According to the content of the sample text, directly summarize the current sample text to obtain the summary content;

[0178] (2) Based on the content of the summary and the text, determine the problems in the summary content, which need to be described accurately and not expressed in general terms;

[0179] (3) Based on the issues identified in (2), the sample text is re-summarized to obtain a paraphrase summary. Here, the paraphrase summary means that the output of the large language model is easy to understand. In addition to the direct translation of the text content, the context is also considered to obtain a better expression result.

[0180] (4) Generate text labels and label definitions based on the content of the paraphrase summary.

[0181] (5) Output the final label and label definition.

[0182] The specific output form can be expressed as follows:

[0183] ###Meaning

[0184] {Significant result}

[0185] ****

[0186] ###question

[0187] {List of Questions}

[0188] ****

[0189] ### Paraphrase summary content

[0190] {Summary of the content}

[0191] ****

[0192] ###Tag Definition

[0193] {Tag definition result}

[0194] ****

[0195] ###Label

[0196] {Tag results}

[0197] ****

[0198] Now please generate the text labels and label definitions below the layer as required above.

[0199] text

[0200] Stage 5: Determine text pseudo-labels.

[0201] Here, according to the clustering results of stage 2, the label corresponding to the seed text corresponding to each cluster is used as the pseudo label of the text in the cluster.

[0202] Stage 6: Text classifier training.

[0203] Here, the text results (i.e., pseudo labels) annotated in stage 5 are used to divide the training text into a training set, a validation set, and a test set. Classification models such as BERT can be used for model training.

[0204] Stage 7: Determine the test set results.

[0205] Here, the text classification model trained in stage 6 is used for verification on the retained test set. When the accuracy of the test set is less than the accuracy threshold (for example, 0.85), all texts whose prediction results are inconsistent with the pseudo-label results (i.e., incorrectly labeled texts) are collected, the prompt words are reconstructed, and the prompt words and the entire text are input into the large language model to adjust the label of the text and generate a new label for the text.

[0206] The prompt word can be expressed in the form of: "You are a knowledge expert who is good at data analysis. You are good at analyzing based on the real labels and predicted labels of the given text."

[0207] The large language model can use the following steps to improve the text label definition:

[0208] (1) Analyze the text and pseudo-label issues based on pseudo-labels and labels predicted by the model;

[0209] (2) Based on the problem in (1), modify the labeling system and labeling definition;

[0210] (3) Output the final label and label definition.

[0211] The specific output form can be expressed as follows:

[0212] ###question

[0213] {List of Questions}

[0214] ****

[0215] ###Tag Definition

[0216] {Tag definition result}

[0217] ****

[0218] ###Label

[0219] {Tag results}

[0220] ****

[0221] Now please generate the following text labels and label definitions according to the above requirements.

[0222] Text, label, predict

[0223] Stage 8: Input the adjusted text into the classification model for training. When the result of the test set is less than the accuracy threshold, continue to perform stage 7 until the accuracy of the model test set is greater than the accuracy threshold.

[0224] The following continues to describe the exemplary structure of the text classification model training device 543 provided in the embodiment of the present application implemented as a software module. In some embodiments, such as Figure 2A As shown, the software modules stored in the training device 543 of the text classification model in the memory 540 may include: a determination module 5431, a generation module 5432, a processing module 5433 and a training module 5434.

[0225] The determination module 5431 is used to cluster the initialization text to obtain multiple clusters and filter out the seed text of each cluster; the generation module 5432 is used to construct the first prompt word corresponding to the seed text of each cluster, and label the seed text of each cluster based on the first prompt word to obtain the label of the seed text of each cluster; the processing module 5433 is used to use the label of the seed text of each cluster as the pseudo-label of the initialization text; the training module 5434 is used to train the text classification model based on the initialization text carrying the pseudo-label.

[0226] In some embodiments, the determination module 5431 is also used to embed the initialization text to obtain a text vector of the initialization text; cluster the text vector to obtain an initial clustering result; and adjust the initial clustering result based on the silhouette coefficient to obtain multiple clusters, wherein each cluster includes part of the initialization text.

[0227] In some embodiments, for each cluster, the determination module 5431 is further used to determine the center of multiple text vectors included in the cluster; and use the initialization text corresponding to the text vector whose distance from the center is less than a distance threshold as the seed text filtered out from the cluster.

[0228] In some embodiments, the determination module 5431 is also used to randomly extract N initialization texts from the initialization texts corresponding to the text vectors whose distance from the center is less than a distance threshold, as seed texts screened from the cluster, where N is a positive integer greater than or equal to 1.

[0229] In some embodiments, the generation module 5432 is further used to fill the first prompt word and the seed text of each cluster into a preset template to obtain second prompt information, wherein the first prompt word is used to indicate the generation of a label for the seed text of each cluster; based on the second prompt information, the large language model is used to perform generation processing to obtain the label of the seed text of each cluster.

[0230] In some embodiments, when there are multiple seed texts, the generation module 5432 is also used to splice the multiple seed texts of each cluster to obtain a spliced ​​text; based on the first prompt word and the spliced ​​text, a large language model is used to generate and process to obtain a label of the spliced ​​text; and the label of the spliced ​​text is used as the label of the multiple seed texts of each cluster.

[0231] In some embodiments, for each cluster, the processing module 5433 is further configured to use the label of the seed text corresponding to the cluster as a pseudo label of the partial initialization text included in the cluster.

[0232] In some embodiments, the training module 5434 is also used to divide the initialization text carrying the pseudo-label into a training set and a test set; train the text classification model based on the training set, and test the accuracy of the trained text classification model based on the test set; when the test result indicates that the accuracy of the trained text classification model is less than the accuracy threshold, obtain the incorrectly labeled text in the test set, wherein the incorrectly labeled text is the initialization text whose classification result output by the trained text classification model is inconsistent with the pseudo-label carried; construct a second prompt word based on the incorrectly labeled text, and label the initialization text based on the second prompt word to obtain the pseudo-label regenerated for the initialization text; train the text classification model based on the initialization text carrying the regenerated pseudo-label.

[0233] In some embodiments, the training module 5434 is also used to fill the second prompt word and the initialization text into a preset template to obtain a first prompt information, wherein the second prompt word is used to indicate the reason for the analysis inconsistency and to regenerate a pseudo-label for the initialization text; based on the first prompt information, a generation process is performed through a large language model to obtain a pseudo-label regenerated for the initialization text.

[0234] In some embodiments, Figure 2B As shown, the software modules stored in the text classification device 643 of the memory 640 may include: an acquisition module 6431 and a classification module 6432 .

[0235] The acquisition module 6431 is used to acquire the text to be classified; the classification module 6432 is used to classify the text to be classified through a text classification model to obtain a classification result of the text to be classified, wherein the text classification model is trained according to a training device of the text classification model.

[0236] It should be noted that the description of the apparatus in the embodiment of the present application is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment, so it will not be repeated. Figure 3 , Figure 4 , Figure 5 , Fig. 6A , Figure 6B , Fig. 7A ,or Figure 7B The present invention should be understood by referring to the description of any one of the accompanying drawings.

[0237] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the computer device executes the training method of the text classification model described in the embodiment of the present application.

[0238] The present application embodiment provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the processor will be caused to execute the text classification model training method provided by the present application embodiment, for example, Figure 3 , Figure 4 , Figure 5 , Fig. 6A , Figure 6B , Fig. 7A ,or Figure 7B The training method of the text classification model is shown.

[0239] In some embodiments, the computer-readable storage medium may be a ferroelectric random access memory (FRAM), ROM, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disk, or compact disc read-only memory (CD-ROM) and other memories; it may also be various devices including one or any combination of the above memories.

[0240] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0241] As an example, the executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0242] In summary, the embodiments of the present application have the following beneficial effects:

[0243] (1) By constructing prompt words and automatically generating labels for seed texts through a large language model, the high reliance on manpower and expertise in the training process of text classification models can be reduced, thereby reducing training costs.

[0244] (2) Since the labels generated by the large language model are high-quality labels, the trained text classification model has a high accuracy rate. The trained text classification model can be used to quickly and accurately classify the classified text.

[0245] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A training method for a text classification model, characterized in that: The method comprises: Cluster the initial text to obtain multiple clusters, and filter out the seed text of each cluster; Constructing a first prompt word corresponding to the seed text of each cluster, and labeling the seed text of each cluster based on the first prompt word to obtain a label of the seed text of each cluster; Using the label of the seed text as the pseudo label of the initialization text; A text classification model is trained based on the initialization text carrying the pseudo label.

2. The method according to claim 1, characterized in that The training of the text classification model based on the initialization text carrying the pseudo label includes: Dividing the initialized text carrying the pseudo-label into a training set and a test set; Training the text classification model based on the training set, and testing the accuracy of the trained text classification model based on the test set; When the test result indicates that the accuracy of the trained text classification model is less than the accuracy threshold, obtaining the wrongly annotated text in the test set, wherein the wrongly annotated text is the initialization text whose classification result output by the trained text classification model is inconsistent with the pseudo label carried; Constructing a second prompt word based on the incorrectly annotated text, and labeling the initialization text based on the second prompt word to obtain a pseudo label regenerated for the initialization text; The text classification model is trained based on the initialization text carrying the regenerated pseudo-label.

3. The method according to claim 2, characterized in that The labeling process of the initialization text based on the second prompt word to obtain a pseudo label regenerated for the initialization text includes: Filling the second prompt word and the initialization text into a preset template to obtain first prompt information, wherein the second prompt word is used to indicate the reason for the inconsistent analysis and to regenerate a pseudo label of the initialization text; Based on the first prompt information, a generation process is performed through a large language model to obtain a pseudo label regenerated for the initialization text.

4. The method according to claim 1, characterized in that: The initialization text is clustered to obtain multiple clusters, including: Performing embedding processing on the initialization text to obtain a text vector of the initialization text; Clustering the text vectors to obtain an initial clustering result; The initial clustering result is adjusted based on the silhouette coefficient to obtain a plurality of clusters, wherein each cluster includes a portion of the initialization text.

5. The method according to claim 4, characterized in that The seed text of each cluster is screened out, including: The following processing is performed for each cluster: Determine the center of the plurality of text vectors included in the cluster; The initialization text corresponding to the text vector whose distance from the center is less than the distance threshold is used as the seed text selected from the cluster.

6. The method according to claim 5, characterized in that The initialization text corresponding to the text vector whose distance from the center is less than the distance threshold is used as the seed text screened from the cluster, including: From the initialization texts corresponding to the text vectors whose distance from the center is less than a distance threshold, N initialization texts are randomly extracted as seed texts screened from the cluster, where N is a positive integer greater than or equal to 1.

7. The method according to claim 1, characterized in that The step of using the label of the seed text as the pseudo label of the initialization text includes: The following processing is performed for each of the clusters: The label of the seed text corresponding to the cluster is used as a pseudo label of the portion of the initialization text included in the cluster.

8. The method according to any one of claims 1 to 7, characterized in that: The step of labeling the seed text of each cluster based on the first prompt word to obtain a label of the seed text of each cluster includes: Filling the first prompt word and the seed text of each cluster into a preset template to obtain second prompt information, wherein the first prompt word is used to indicate a label for generating the seed text of each cluster; Based on the second prompt information, a generation process is performed through a large language model to obtain a label of the seed text of each cluster.

9. The method according to any one of claims 1 to 7, characterized in that: When there are multiple seed texts, labeling the seed texts of each cluster based on the first prompt word to obtain a label of the seed text of each cluster includes: Splicing the multiple seed texts of each cluster to obtain a spliced ​​text; Based on the first prompt word and the concatenated text, a generation process is performed by a large language model to obtain a label of the concatenated text; The label of the concatenated text is used as the label of the multiple seed texts of each cluster.

10. A text classification method, characterized in that: The method comprises: Get the text to be classified; The text to be classified is classified by a text classification model to obtain a classification result of the text to be classified, wherein the text classification model is trained according to the method according to any one of claims 1 to 9.

11. A training device for a text classification model, characterized in that: The device comprises: A determination module is used to cluster the initialization text to obtain multiple clusters and filter out the seed text of each cluster; A generation module, used to construct a first prompt word corresponding to the seed text of each cluster, and label the seed text of each cluster based on the first prompt word to obtain a label of the seed text of each cluster; A processing module, used for using the label of the seed text as a pseudo label of the initialization text; A training module is used to train a text classification model based on the initialization text carrying the pseudo label.

12. A text classification device, characterized in that: The device comprises: An acquisition module is used to obtain the text to be classified; A classification module is used to classify the text to be classified through a text classification model to obtain a classification result of the text to be classified, wherein the text classification model is trained according to the method according to any one of claims 1 to 9.

13. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, for implementing the text classification model training method described in any one of claims 1 to 9, or implementing the text classification method described in claim 10, when executing the executable instructions stored in the memory.

14. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, the training method of the text classification model described in any one of claims 1 to 9 is implemented, or the text classification method described in claim 10 is implemented.

15. A computer program product comprising a computer program or computer executable instructions, characterized in that: When the computer program or computer executable instructions are executed by a processor, the training method of the text classification model described in any one of claims 1 to 9 is implemented, or the text classification method described in claim 10 is implemented.

Images