Text labeling model acquisition and text labeling method, related apparatus
By training the first and second classifiers and combining low-noise and high-noise sample sets, the text annotation model is optimized, solving the problem of high cost and low efficiency in large language model annotation and achieving efficient and low-cost text annotation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HYTERA COMM CORP
- Filing Date
- 2026-02-13
- Publication Date
- 2026-06-05
AI Technical Summary
Direct annotation of large language models in text classification tasks is costly and inefficient, limiting their widespread application in industrial scenarios.
By training the first and second classifiers, using a low-noise first sample set and a high-noise second sample set, and combining a large language model, the text annotation model is gradually optimized, reducing the dependence on LLM and improving annotation efficiency and accuracy.
The trained text annotation model is highly accurate, low-cost, and efficient in text annotation, reducing reliance on large language models and improving annotation efficiency.
Smart Images
Figure CN122153060A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a method and apparatus for obtaining a text annotation model and annotating text. Background Technology
[0002] In recent years, Large Language Models (LLMs) have demonstrated powerful capabilities in text classification tasks. However, directly using LLMs to classify and label massive amounts of unlabeled text suffers from high costs and low efficiency, limiting their widespread application in industrial scenarios. Summary of the Invention
[0003] In view of the above problems, this application provides a method and apparatus for obtaining a text annotation model and annotating text, so as to improve annotation efficiency and reduce annotation costs. The specific solution is as follows:
[0004] The first aspect of this application provides a method for obtaining a text annotation model, including:
[0005] The first classifier and the second classifier of the first annotation model are trained using the first sample set, and the second classifier of the first annotation model is trained using the second sample set to obtain the second annotation model. The noise level of the first sample set is lower than that of the second sample set. The noise level of any text sample set is inversely correlated with the accuracy of the text's category label. The first classifier is used to predict the category of the text, and the second classifier is used to predict the keywords of the text.
[0006] Using the second annotation model, keywords for the text in the third sample set are obtained;
[0007] By inputting the third sample set and the keywords into a large language model (LLM), the category labels of the text in the third sample set output by the LLM are obtained;
[0008] Based on the third sample set and the category labels of the text in the third sample set, the second annotation model is trained to obtain the text annotation model.
[0009] In one possible implementation, before training the first classifier and the second classifier of the first labeled model using the first sample set, and training the second classifier of the first labeled model using the second sample set, the method further includes:
[0010] By inputting the text sample to be labeled into the LLM, the sample category label and sample keywords of the text sample to be labeled are obtained from the output of the LLM;
[0011] Using the text to be labeled, the category labels and keywords of the text samples to be labeled, an initial model is trained to obtain the first labeling model, which includes the first classifier and the second classifier.
[0012] In one possible implementation, the first loss function used to train the initial model is determined based on classification loss and keyword loss, wherein the classification loss is obtained based on the sample category labels and the predicted category labels of the initial model's output of the unlabeled text, and the keyword loss is obtained based on the sample keywords and the predicted keywords of the initial model's output of the unlabeled text.
[0013] In one possible implementation, the process of obtaining the first sample set and the second sample set includes:
[0014] By inputting the first labeled data and the loss corresponding to the first labeled data into the second model, the second model outputs a first sample set and a second sample set based on a loss threshold. The first labeled data includes the text to be labeled, the category label of the text sample to be labeled, and keywords. The loss corresponding to the first labeled data includes the loss of training the initial model using the first labeled data.
[0015] In one possible implementation, the second loss function used to train the second model is obtained by summing the products of the loss of the third sample set and the loss of the fourth sample set with preset coefficients, wherein the noise level of the third sample set is lower than that of the fourth sample set, and the preset coefficients are in the range of (0,1).
[0016] A second aspect of this application provides a text annotation method, comprising:
[0017] By inputting text into a text annotation model, the category labels of the text output by the text annotation model are obtained, wherein the text annotation model is obtained by the method described in the first aspect of this application.
[0018] A third aspect of this application provides a text annotation apparatus, comprising: a module that runs the method of the first aspect of this application or any implementation thereof or the second aspect thereof.
[0019] A fourth aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the first aspect or any implementation thereof or the method of the second aspect.
[0020] A fifth aspect of this application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:
[0021] The memory is used to store computer programs;
[0022] The processor is used to execute the computer program so that the electronic device can implement the first aspect or any implementation of the first aspect or the method of the second aspect.
[0023] The sixth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to perform the first aspect or any implementation thereof or the method of the second aspect.
[0024] By employing the above technical solution, the text annotation model acquisition and text annotation method and related apparatus provided in this application train the first classifier and second classifier of the first annotation model using a first sample set, and train the second classifier of the first annotation model using a second sample set to obtain the second annotation model. Because the first classifier is used to predict the category of the text and the second classifier is used to predict the keywords of the text, it is beneficial to improve the role of keywords in training and achieve the purpose of training the model from two perspectives. Furthermore, because the noise level of the first sample set is lower than that of the second sample set, and the noise level of any text sample set is inversely correlated with the accuracy of the text category label, the adverse effect of noise on classification is reduced. Using the second annotation model, the keywords of the text in the third sample set are obtained. By inputting the third sample set and keywords into a large language model (LLM), the category labels of the text in the third sample set output by the LLM are obtained. Based on the third sample set and the category labels of the text in the third sample set, the second annotation model is trained to obtain the text annotation model. It can be seen that the second annotation model can learn the capabilities of the LLM without having to use the LLM again, thus reducing costs. In summary, the text annotation model trained by this method has high accuracy and low usage cost. Attached Figure Description
[0025] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0026] Figure 1 A schematic diagram of the system architecture of a text annotation system provided for embodiments of this application;
[0027] Figure 2 A flowchart illustrating a method for obtaining a text annotation model, provided for an embodiment of this application;
[0028] Figure 3An example structural diagram of a text annotation device provided for embodiments of this application;
[0029] Figure 4 This is a structural example diagram of an electronic device provided for an embodiment of this application. Detailed Implementation
[0030] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.
[0031] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.
[0032] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.
[0033] See Figure 1 , Figure 1 A schematic diagram of the system architecture for a text annotation system is shown. The system may include a first server 100, a second server 200, and a terminal 300.
[0034] The first server 100 runs an LLM model, and the second server 200 runs a text annotation model. In order to reduce the cost of text annotation, the text annotation model running on the second server 200 is trained based on the LLM model running on the first server 100.
[0035] The terminal can annotate text by calling the text annotation model.
[0036] The text annotation model can be set to a smaller model compared to LLM, such as a small language model. Based on this, the cost of annotation can be reduced and the efficiency of annotation can be improved compared to terminal calling LLM for text annotation.
[0037] The terminal 300 in this application embodiment can be a mobile phone, tablet computer, wearable device, vehicle-mounted device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.
[0038] Server 100 or 200 includes a bus, processor, communication interface, and memory. The processor, memory, and communication interface communicate with each other via the bus.
[0039] The memory can be used to store the software code related to the text annotation model acquisition method and text annotation method provided in the following embodiments. The processor can execute the steps of the method provided in the following embodiments, and can also schedule other units to implement the corresponding functions.
[0040] Figure 2 This is a flowchart of a method for obtaining a text annotation model provided in an embodiment of this application, including the following steps:
[0041] S1. Obtain the first labeled dataset.
[0042] From large-scale unlabeled datasets random sampling subset Get subset The labeled data will be subsets And its labeled data, as the first labeled dataset.
[0043] In some implementations, prompt words are designed, and prompt words and subsets are combined. Input an LLM and get a subset of the LLM output. The labeled data.
[0044] For example, the prompt includes information guiding the output to include: (a) a category label; and (b) a set of keywords highly relevant to the category label. Based on this prompt, the LLM can output a subset of the keywords. Each piece of data contains a text category label and a set of keywords that are highly relevant to the category label.
[0045] It should be noted that although the prompt words indicate a set of keywords that are highly related to the category labels, it is possible that the keywords provided by the LLM are not highly related to the category labels provided by the LLM. In this embodiment, such keywords are also usable. That is to say, the degree of relevance between the keywords and the category labels is not a limitation.
[0046] It is understandable that any labeled data in the first labeled dataset includes: text, the category label of the text, and a set of keywords that are highly related to the category label.
[0047] In some implementations, to improve the reliability of the first labeled dataset, the unlabeled dataset is used multiple times. A subset of samples is randomly selected, and the labeled data of this subset is obtained. Multiple subsets and their labeled data are then merged to obtain the first labeled dataset. In other words, a self-consistent strategy (multiple sampling and union) is used to improve the reliability of keywords and obtain the first labeled dataset. .
[0048] S2. Train small language models (SLMs) using the first labeled dataset to obtain the first SLMs.
[0049] In some implementations, SLMs include a feature encoder, a first classifier, and a second classifier.
[0050] For example, the feature encoder is a Bidirectional Encoder Representations from Transformers (BERT) encoder. The feature encoder is used to encode the text in the first labeled dataset of the input to obtain a Classification Token (CLS) vector.
[0051] The function of the first classifier is to output a class prediction result based on the CLS vector output by the BERT encoder. Specifically, the first classifier is a binary classification model. The input is the CLS vector, and the output is the predicted possible class of the CLS vector and whether the class belongs to that class. Therefore, based on the output of the first classifier, class information, such as the class label, can be determined. For example, the first classifier is also used to determine class information, or the class information can be determined by other modules.
[0052] The function of the second classifier is to output the keyword results of the text based on the CLS vector output by the BERT encoder. Specifically, the second classifier is a binary classification model that takes the CLS vector as input and outputs the predicted words based on the CLS vector, and whether the word is retained as a keyword. Furthermore, keywords can be determined based on the output of the second classifier. For example, the second classifier is also used to determine keywords, or keywords can be determined by other modules.
[0053] It is evident that the first classifier performs sentence-level classification, while the second classifier performs word-level classification.
[0054] An example of the loss function used to train SLMs is: It is the classification loss (i.e., the loss of the first classifier). It is the keyword loss (i.e., the loss of the second classifier).
[0055] Based on the structure and loss function of SLMs described above, the specific process of training SLMs using the first labeled dataset includes: inputting any labeled data (text, text category label, and a set of keywords highly related to the category label) from the first labeled dataset into the SLMs; the feature encoder encodes the text in the labeled data to obtain a CLS vector; based on the information output by the first classifier, the category label of the text in the labeled data is determined (referred to as the predicted category label); based on the information output by the second classifier, the keywords of the text in the labeled data are determined (referred to as the predicted keywords); based on the category label and the predicted category label of the labeled data, the following is obtained: Based on the keywords and predicted keywords in this labeled data, we obtain .based on and The loss function is obtained. Then, the loss function converges, achieving the goal of training SLMs.
[0056] In this step, a model that includes a first classifier and a second classifier is innovatively proposed, which is beneficial to improve the role of keywords in training and achieve the goal of training the model from two perspectives.
[0057] The process in this embodiment is an iterative training process. For the sake of distinction, the model trained in this step is called the first SLMs.
[0058] The second round of training for SLMs will now begin:
[0059] S3. Based on the loss of training SLMs on the first labeled dataset, the first labeled dataset is divided into a clean sample set and a noisy sample set.
[0060] For example, a Gaussian Mixture Model (GMM) is used to divide the first labeled dataset into a clean sample set and a noisy sample set. Specifically, a threshold is input into the GMM, so that the GMM outputs a classification result for any dataset in the input first labeled dataset based on the threshold. The classification result indicates whether the dataset is a clean sample set or a noisy sample set.
[0061] For example, the loss function of GMM is as follows:
[0062]
[0063] That's the total loss. It is the loss from training on a clean sample set. It is the loss from training on a noisy sample set. These are loss weight parameters that can be automatically learned to balance. and , It can be pre-configured, with a range of (0,1). It is a clean sample set. It is a noise sample set.
[0064] Using GMM to divide the clean sample set and the noisy sample set, the automatic division is achieved without manual intervention.
[0065] S4. Use clean sample sets and noisy sample sets to train the first SLMs respectively, and obtain the second SLMs.
[0066] For the specific process of training the first SLMs using a clean sample set, please refer to the process of training SLMs using the first labeled dataset in S2, which will not be repeated here.
[0067] As for the noisy sample set, assuming that the keyword annotation is more reliable, and since the noise may be caused by the inaccuracy of the class labels output by the LLM, we only use the noisy sample set to train the second classifier used to predict keywords in the first SLMs, and do not train the first classifier. This is equivalent to shielding the adverse effects of classification noise on the first classifier. Specifically, compared with the process of training SLMs using the first labeled dataset in S2, we only perform the part related to the second classifier and not the part related to the first classifier.
[0068] In other words, the first SLMs learn the classification labels and keywords of the clean sample set, which is a dual-task supervised training method, while the first SLMs learn the keywords of the noisy sample set, thereby reducing the adverse effects of the classification noise of the LLM on the first SLMs.
[0069] In some implementations, the first SLMs are trained using a clean sample set (or a noisy sample set) to obtain an intermediate model, and then the intermediate model is trained using a noisy sample set (or a clean sample set) to obtain a second SLM.
[0070] In other implementations, the first SLMs are trained using a clean sample set to obtain a first intermediate model, the first SLMs are trained using a noisy sample set to obtain a second intermediate model, and the first and second intermediate models are then fused to obtain the second SLMs.
[0071] The third round of training for SLMs will now begin:
[0072] S5, Using the second SLMs Extract keywords from the remaining unlabeled data D_rest.
[0073] S6. Using LLM, obtain the second labeled dataset for the text and corresponding keywords in D_rest.
[0074] For example, the text and the keywords extracted from the text are compressed to obtain a compressed file, and the compressed file is sent to the LLM to improve the efficiency of transmission to the LLM.
[0075] Understandably, compared to the method in S1 where LLM directly outputs category labels from text, this step uses keywords corresponding to the text as guidance, which helps LLM output category labels more efficiently.
[0076] In this step, LLM can output keywords or not; there is no restriction here.
[0077] S7. Using the first and second labeled datasets, train the second SLMs to obtain the third SLMs.
[0078] For the specific training process in this step, please refer to S2, which will not be repeated here.
[0079] It is understandable that each of the three training rounds can be iterated multiple times to gradually improve the model performance.
[0080] The text annotation model acquisition method provided in this implementation learns text annotation capabilities from LLM and undergoes multiple rounds of iterative training. Therefore, the trained text annotation model has better generalization ability and can output annotation results (category labels) with higher accuracy. Compared with LLM, the text annotation model is smaller in scale and can be deployed online, resulting in lower training and application costs and higher training and application efficiency. Using keywords as a guiding factor in training, keyword extraction and classification tasks are co-optimized during the training phase, which can improve the efficiency of LLM model output samples. In summary, the text annotation model acquisition method provided in this implementation is an efficient training method.
[0081] Accordingly, embodiments of this application also provide a text annotation method, including: inputting text into a text annotation model to obtain category labels of the text output by the text annotation model, wherein the text annotation model is obtained through the method provided in the above embodiments. For example, in conjunction with... Figure 1 The system shown, Figure 2The method shown can be applied to the second server 200 to train the text annotation model. After training is completed, the text annotation model can be deployed online on the second server 200. The terminal 300 does not need to call the LLM on the first server 100, but can call the text annotation model on the second server 200 to annotate the text. Compared with calling the LLM, it is more efficient and less costly.
[0082] Embodiments of this application also provide a text annotation apparatus, including a module that runs the above-described text annotation method and / or text annotation model acquisition method.
[0083] For example, such as Figure 3 As shown, it includes: a text annotation module and a text annotation model acquisition module.
[0084] The text annotation module is used to call the text annotation model after obtaining the text and output the category labels of the text.
[0085] The text annotation model acquisition module is used to train a first classifier and a second classifier of the first annotation model using a first sample set, and to train the second classifier of the first annotation model using a second sample set to obtain a second annotation model. The noise level of the first sample set is lower than that of the second sample set. The noise level of any text sample set is inversely correlated with the accuracy of the text's category label. The first classifier is used to predict the category of the text, and the second classifier is used to predict the keywords of the text. Using the second annotation model, the keywords of the text in the third sample set are obtained. By inputting the third sample set and the keywords into a large language model (LLM), the category labels of the text in the third sample set output by the LLM are obtained. Based on the third sample set and the category labels of the text in the third sample set, the second annotation model is trained to obtain the text annotation model.
[0086] Optionally, the text annotation model acquisition module is further configured to, before training the first classifier and the second classifier of the first annotation model using the first sample set and training the second classifier of the first annotation model using the second sample set, obtain the sample category labels and sample keywords of the text samples to be annotated by inputting the text samples to be annotated into the LLM; train an initial model using the text to be annotated, the category labels and keywords of the text samples to be annotated, to obtain the first annotation model, wherein the initial model includes the text obtained from the first classifier and the second classifier in the fourth sample set.
[0087] Optionally, the first loss function used to train the initial model is determined based on classification loss and keyword loss. The classification loss is obtained based on the sample category label and the predicted category label of the initial model's output of the unlabeled text, and the keyword loss is obtained based on the sample keywords and the predicted keywords of the initial model's output of the unlabeled text.
[0088] Optionally, the process of the text annotation model acquisition module to acquire the first sample set and the second sample set includes: inputting the first annotation data and the loss corresponding to the first annotation data into the second model to obtain the first sample set and the second sample set output by the second model based on the loss threshold. The first annotation data includes the text to be annotated, the category label and keywords of the text sample to be annotated, and the loss corresponding to the first annotation data includes the loss of training the initial model using the first representation data.
[0089] Optionally, the second loss function used to train the second model is obtained by summing the products of the loss of the third sample set and the loss of the fourth sample set with preset coefficients, wherein the noise level of the third sample set is lower than that of the fourth sample set, and the preset coefficients are in the range of (0,1).
[0090] This device can provide highly accurate labeling at a low cost.
[0091] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the text annotation model acquisition methods or text annotation methods provided in this application.
[0092] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the text annotation model acquisition methods or text annotation methods provided in this application.
[0093] Embodiments of this application also provide an electronic device, which can be... Figure 1 The second server 200 in the middle, such as Figure 4As shown, the device specifically includes a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0094] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, memory cards, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.
[0095] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.
[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0097] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.
[0098] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
Claims
1. A method for obtaining a text annotation model, characterized in that, include: The first classifier and the second classifier of the first annotation model are trained using the first sample set, and the second classifier of the first annotation model is trained using the second sample set to obtain the second annotation model. The noise level of the first sample set is lower than that of the second sample set. The noise level of any text sample set is inversely correlated with the accuracy of the text's category label. The first classifier is used to predict the category of the text, and the second classifier is used to predict the keywords of the text. Using the second annotation model, keywords for the text in the third sample set are obtained; By inputting the third sample set and the keywords into a large language model (LLM), the category labels of the text in the third sample set output by the LLM are obtained; Based on the third sample set and the category labels of the text in the third sample set, the second annotation model is trained to obtain the text annotation model.
2. The method according to claim 1, characterized in that, Before training the first classifier and the second classifier of the first labeled model using the first sample set, and training the second classifier of the first labeled model using the second sample set, the method further includes: By inputting the text sample to be labeled into the LLM, the sample category label and sample keywords of the text sample to be labeled are obtained from the output of the LLM; Using the text sample to be labeled, the category label and keywords of the text sample to be labeled, an initial model is trained to obtain the first labeling model, which includes the first classifier and the second classifier.
3. The method according to claim 2, characterized in that, The first loss function used to train the initial model is determined based on classification loss and keyword loss. The classification loss is obtained based on the sample category label and the predicted category label of the initial model's output of the unlabeled text, and the keyword loss is obtained based on the sample keywords and the predicted keywords of the initial model's output of the unlabeled text.
4. The method according to claim 2 or 3, characterized in that, The process for obtaining the first sample set and the second sample set includes: By inputting the first labeled data and the loss corresponding to the first labeled data into the second model, the second model outputs a first sample set and a second sample set based on a loss threshold. The first labeled data includes the text to be labeled, the category label of the text sample to be labeled, and keywords. The loss corresponding to the first labeled data includes the loss of training the initial model using the first labeled data.
5. The method according to claim 4, characterized in that, The second loss function used to train the second model is obtained by multiplying the loss of the third sample set and the loss of the fourth sample set by a preset coefficient. The noise level of the third sample set is lower than that of the fourth sample set, and the preset coefficient ranges from (0,1).
6. A text annotation method, characterized in that, include: By inputting text into a text annotation model, the category labels of the text output by the text annotation model are obtained, wherein the text annotation model is obtained by the method described in any one of claims 1-5.
7. A text annotation device, characterized in that, Includes a module that performs the method according to any one of claims 1-6.
8. A computer program product, characterized in that, Includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 6.
9. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is configured to execute the computer program to enable the electronic device to implement the method as described in any one of claims 1 to 6.
10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to perform the method as described in any one of claims 1 to 6.