Text tagging method, device, equipment and storage medium based on active learning

Abstract

The present application relates to the field of computer technology, and in particular to the field of natural language processing technology, and provides a text tagging method, device, equipment and storage medium based on active learning, which are used to improve the accuracy and efficiency of text tagging. When the method iteratively trains the text classification model based on the text sample pool, in addition to using the marked text itself, it also uses the real explanation of the causal relationship between the pre-annotated representation text and the label, so that the model can accurately understand the deep semantic information of the text, reduce the model's dependence on the surface semantic meaning of the text, and improve the generalization and accuracy of label prediction; at the same time, it also uses the loss caused by the real explanation of each marked text and the corresponding evaluation value, so that the model can accurately evaluate the explanation, so as to select multiple candidate texts with richer category information for annotation, thereby improving the richness of the text category while reducing the resources occupied by the annotation, and improving the efficiency of text annotation.

Description

Background Art

[0002] In natural language processing tasks, the training of text classification models requires a large amount of labeled data. However, most of the data pools from which text samples come are massive amounts of unlabeled data, and some data contain less classification information. If the categories are manually labeled one by one, errors may occur due to subjective factors, thus affecting the accuracy of text classification.

[0003] With the development of machine learning and artificial intelligence technology, active learning can effectively improve the quality and efficiency of text tagging and has been widely studied in natural language processing tasks. At present, there are three methods for labeling text based on active learning:

[0004] 1. Selection strategy based on information

[0005] This method tends to select uncertain and repetitive data without information from the data pool, which results in low accuracy of text labels and affects the accuracy of text classification model training.

[0006] 2. Selection strategy based on representativeness

[0007] This method relies on the input of the model, such as the vector representation of text. Usually, the data selected from the data pool is relatively simple, resulting in a relatively single category label, which affects the accuracy of text classification model training.

[0008] 3. Hybrid Selection Strategy

[0009] This method improves the annotation quality of text category labels by integrating multiple data selection strategies, but the model has more parameters and the training is more complicated. Summary of the invention

[0010] The embodiments of the present application provide a text tagging method, apparatus, device and storage medium based on active learning, so as to improve the accuracy of understanding the meaning of text and efficiently obtain high-quality labeled text for training text classification models.

[0011] The present application provides a text tagging method based on active learning, including:

[0012] A text classification model is trained for multiple rounds based on a pre-built text sample pool; wherein the text sample pool includes a labeled text subset and an unlabeled text subset, and each iterative training includes:

[0013] For the selected multiple marked texts, the following operations are performed respectively: using the text classification model, based on the text features of the marked texts, obtaining predicted text labels of the marked texts;

[0014] Based on the obtained predicted text labels, the classification loss is calculated by combining the pre-annotated real text labels, real explanations and corresponding evaluation values, where each explanation is used to characterize the causal relationship between the label and the text;

[0015] The parameters of the text classification model are adjusted according to the classification loss, and the adjusted text classification model is used to obtain the predicted text labels and predicted explanations of the selected multiple unlabeled texts respectively, and based on the predicted text labels and the predicted explanations, a plurality of candidate texts to be labeled are screened out from the multiple unlabeled texts to update the labeled text subset and the unlabeled text subset.

[0016] An embodiment of the present application provides a text marking device based on active learning, comprising:

[0017] A sample construction module, used for performing multiple rounds of iterative training on a text classification model based on a pre-constructed text sample pool; wherein the text sample pool includes a labeled text subset and an unlabeled text subset;

[0018] The label prediction module is used to perform the following operations for the selected multiple labeled texts during each iteration of training: using the text classification model to obtain the predicted text label of the labeled text based on the text features of the labeled text;

[0019] A loss calculation module, used to calculate the classification loss based on the obtained predicted text labels, combined with the pre-labeled real text labels, the real explanations and the corresponding evaluation values, wherein each explanation is used to characterize the causal relationship between the label and the text;

[0020] The text selection module is used to adjust the parameters of the text classification model according to the classification loss, use the adjusted text classification model to obtain the predicted text labels and predicted explanations of the selected multiple unlabeled texts, and based on the predicted text labels and predicted explanations, screen out multiple candidate texts to be labeled from the multiple unlabeled texts to update the labeled text subset and the unlabeled text subset.

[0021] Optionally, the text classification model includes a bidirectional encoder, and the label prediction module is specifically used for:

[0022] After adding a classification mark to the marked text, the marked text is input into the bidirectional encoder to extract text features of the marked text, and based on the text features, a context sentence representation of the marked text is obtained;

[0023] Based on the context sentence representation, the probability distribution of the marked text in the label space is calculated, and based on the probability distribution, the predicted text label of the marked text is obtained.

[0024] Optionally, the text classification model includes a bidirectional encoder and a unidirectional decoder, and the loss calculation module is specifically used for:

[0025] Using the bidirectional encoder, based on the predicted text labels and the true text labels, obtaining the label prediction loss of the text classification model;

[0026] For the multiple marked texts, the one-way decoder is used to perform the following operations respectively: based on the target interpretation selected from each true interpretation corresponding to the marked text and the context sentence representation of the marked text, a sub-interpretation generation loss of the marked text is obtained, and based on the evaluation values ​​of each true interpretation corresponding to the marked text, two-by-two combinations are performed to obtain the sub-interpretation ranking loss of the marked text;

[0027] Based on the obtained sub-explanation generation losses, obtaining the explanation generation loss of the text classification model;

[0028] Based on the obtained sub-explanation ranking losses, obtaining the explanation ranking loss of the text classification model;

[0029] A classification loss is calculated based on the label prediction loss, the explanation generation loss, and the explanation ranking loss.

[0030] Optionally, the loss calculation module is specifically used to:

[0031] According to the evaluation values ​​of the real interpretations of the marked text from large to small, a target interpretation is selected from the real interpretations, and a target text length of the target interpretation is obtained;

[0032] A sub-interpretation generation loss of the marked text is obtained based on the target interpretation, a context sentence representation of the marked text, and the target text length.

[0033] Optionally, each true interpretation of the marked text includes a reasonable interpretation and an unreasonable interpretation, and the loss calculation module is specifically used to:

[0034] For each true interpretation of the marked text, the following operations are performed respectively: normalization is performed based on the text length of the true interpretation to obtain a conditional logarithmic probability, where the conditional logarithmic probability is used to represent the rationality of the true interpretation;

[0035] Based on the evaluation values ​​of the true interpretations of the marked text, the conditional logarithmic probabilities of the true interpretations are combined in pairs, and the difference between the two conditional logarithmic probabilities in each combination is calculated;

[0036] The obtained differences are compared with preset values ​​respectively, and the sub-interpretation sorting loss is determined according to the comparison result.

[0037] Optionally, the text classification model includes a bidirectional encoder and a unidirectional decoder, and the text selection module is specifically used for:

[0038] For the multiple unlabeled texts, the following operations are respectively performed: using the adjusted bidirectional decoder, based on the text features of the unlabeled text, obtaining the context sentence representation of the unlabeled text, and based on the context sentence representation, obtaining the information entropy of the unlabeled text in the label space and normalizing it;

[0039] generating, using the unidirectional decoder, at least one predicted interpretation of the unlabeled text based on the predicted text label of the unlabeled text and the context sentence representation;

[0040] Based on the text length of the at least one predicted interpretation, obtaining a corresponding conditional log probability;

[0041] Selecting a candidate explanation from the at least one predicted explanation in descending order of the obtained conditional logarithmic probabilities, and normalizing the conditional logarithmic probability of the candidate explanation;

[0042] Weighting the normalized information entropy and conditional logarithmic probability to obtain a ranking index of the unlabeled text;

[0043] A plurality of candidate texts to be marked are selected from the plurality of unmarked texts in descending order of the sorting indicators.

[0044] Optionally, the text selection module is specifically used for:

[0045] Using the unidirectional decoder in the text classification model, based on the text features of the unlabeled text, a plurality of word units and corresponding word probabilities are generated;

[0046] Selecting a preset number of candidate word units in descending order of probability of each word;

[0047] Based on the multiple candidate word-grams, a phrase probability distribution of the next word-gram sequence is obtained, and the multiple candidate word-gram sequences of the preset cluster number are selected in descending order of the probability of each phrase,

[0048] The word supplement process of each candidate word-gram sequence is repeated until a stop condition is satisfied, thereby obtaining the at least one predicted interpretation.

[0049] Optionally, the device further includes a business processing module, and after the text classification model training is completed, the business processing module is used to:

[0050] Get the target text of the target business;

[0051] Using a trained text classification model to obtain a target category of the target text;

[0052] Based on the target category, performing business processing on the target text;

[0053] The target service includes at least one of text translation, email filtering, voice query, article recommendation, and news classification.

[0054] An electronic device provided by an embodiment of the present application includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the above-mentioned text labeling methods based on active learning.

[0055] An embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to perform any step of the above-mentioned text tagging method based on active learning.

[0056] An embodiment of the present application provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of any one of the above-mentioned text tagging methods based on active learning.

[0057] The text tagging method, apparatus, device and storage medium based on active learning provided by the embodiments of the present application have the following beneficial effects:

[0058] For the text classification task in natural language processing, the text classification model is iteratively trained based on a text sample pool containing a labeled text subset and an unlabeled text subset. In addition to using the labeled text itself, the pre-labeled real explanation that represents the causal relationship between each labeled text and the real text label is also used, so that the model can accurately understand the deep semantic information of the text, reduce the model's dependence on the surface semantic meaning of the text, and improve the generalization and accuracy of label prediction; at the same time, when calculating the classification loss, in addition to using each predicted text label and each real text label, the pre-labeled real explanation and the corresponding evaluation value are also used. In this way, the text classification model adjusted based on the classification loss can not only optimize the uncertainty information of label prediction, but also evaluate the explanation that represents the causal relationship between the label and the text, so as to select multiple candidate texts with richer category information from the unlabeled text subset for annotation, thereby improving the richness of text categories while reducing the resources occupied by annotation, thereby improving the accuracy of text classification tasks.

[0059] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0061] Figure 1 An optional schematic diagram of an application scenario in an embodiment of the present application;

[0062] Figure 2 This is an architecture diagram of a text classification model based on explainable active learning;

[0063] Figure 3 A network architecture diagram of a text classification model provided in an embodiment of the present application;

[0064] Figure 4 A case study of the marked text provided for the embodiment of the present application;

[0065] Figure 5 A flowchart of a text labeling method based on active learning is provided for an embodiment of the present application;

[0066] Figure 6 A flowchart of text label prediction is provided for an embodiment of the present application;

[0067] Fig. 7A A calculation flow chart of classification loss is provided for an embodiment of the present application;

[0068] Figure 7B A schematic diagram of classified loss content is provided for an embodiment of the present application;

[0069] Figure 8 A calculation flow chart for explaining the generation loss is provided for the embodiment of the present application;

[0070] Fig. 9 A calculation flow chart for explaining sorting loss is provided for the embodiment of the present application;

[0071] Fig.10 A flowchart for selecting unmarked text is provided for an embodiment of the present application;

[0072] Fig.11 A schematic diagram of an unlabeled text prediction process is provided for an embodiment of the present application;

[0073] Fig.12 A flowchart for generating an explanation is provided for an embodiment of the present application;

[0074] Fig.13A Provide a schematic diagram for explaining generation of an embodiment of the present application;

[0075] Fig. 13B A schematic diagram of the labeling result of unlabeled text provided in an embodiment of the present application;

[0076] Fig.14 A schematic diagram of experimental results provided for the embodiments of the present application;

[0077] Fig.15 A schematic diagram of a text marking device based on active learning provided in an embodiment of the present application;

[0078] Fig.16 A schematic diagram of a hardware structure of an electronic device to which an embodiment of the present application is applied;

[0079] Fig.17 The present invention is a schematic diagram of the hardware structure of another electronic device to which the embodiments of the present application are applied. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0081] The following is an introduction to some concepts involved in the embodiments of the present application.

[0082] Active Learning (AL) is a machine learning or artificial intelligence method that actively selects the most valuable samples for labeling. Its purpose is to use as few high-quality labeled samples as possible to achieve the highest performance of the classification model. In other words, under the premise of a limited labeling budget, by increasing the gain of samples and labels, the performance of the classification model is maximized. It is a solution to improve the efficiency and accuracy of model training from the perspective of samples.

[0083] Explainable Active Learning (XAL) is a method that encourages classification models to prove the rationality of their predictions and provides the basis for their output prediction results (i.e., explanations), so as to further study the rationality of the explanations and more accurately obtain the classification value of the samples to be labeled.

[0084] Natural language: usually refers to a language that evolves naturally with culture. For example, English, Chinese, and Japanese are examples of natural languages, while Esperanto is an artificial language, that is, a language created for some specific purpose. However, sometimes all languages ​​used by humans (including the above-mentioned languages ​​that evolve naturally with culture, as well as artificial languages) are considered natural languages, in contrast to artificial languages ​​designed for computers, such as programming languages.

[0085] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0086] Artificial intelligence is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation interaction systems, mechatronics and other technologies; the software technologies of artificial intelligence generally include computer vision technology, natural language processing technology, and machine learning / deep learning and other major directions. With the development and progress of artificial intelligence, artificial intelligence is being studied and applied in many fields, such as common smart homes, smart customer service, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, robots, smart medical care, etc. It is believed that with the further development of future technology, artificial intelligence will be applied in more fields and play an increasingly important role.

[0087] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0088] Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0089] Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Deep learning is the core of machine learning and a technology to achieve machine learning. Machine learning usually includes deep learning, reinforcement learning, transfer learning, inductive learning and other technologies. Deep learning includes mobile visual neural network Mobilenet, convolutional neural network (CNN), deep belief network, recursive neural network, autoencoder, generative adversarial network and other technologies.

[0090] In an embodiment of the present application, a text classification model can be trained for multiple rounds of iterations based on a pre-constructed text sample pool containing a subset of labeled text and a subset of unlabeled text, using natural language processing technology and machine learning / artificial intelligence technology. Specifically, a bidirectional encoder and a unidirectional decoder are constructed based on a neural network, wherein the bidirectional encoder is used to predict the labels of multiple labeled texts, and the uncertainty information of the label prediction is calculated in combination with the real text labels, and the label prediction loss is obtained for parameter adjustment. The unidirectional decoder is used to generate at least one predicted explanation for each unlabeled text and evaluate the predicted explanation, and the loss related to the predicted explanation is obtained in combination with each real explanation for parameter adjustment. Since the explanation is used to characterize the causal relationship between the label and the text, the natural language processing model can accurately understand the deep semantic information of the text, reduce the model's dependence on the surface semantic meaning of the text, and improve the generalization and accuracy of the label prediction. In this way, when the adjusted model is used to select unlabeled texts, the category information of each unlabeled text can be accurately obtained, so that multiple candidate texts with the best value for text classification are selected for annotation, the efficiency and quality of data annotation in the text classification task are improved, and the accuracy of the text classification task in practical applications is improved.

[0091] With the development and progress of artificial intelligence, artificial intelligence is being studied and applied in many fields, such as common smart homes, smart customer service, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, robots, smart medical care, etc. I believe that with the further development of future technology, artificial intelligence will be applied in more fields and play an increasingly important role.

[0092] The following is a brief introduction to the design concept of the embodiment of the present application:

[0093] In the field of natural language processing, deep learning has attracted widespread attention due to its high practical value. It can be applied to low-resource tasks by focusing human labeling work on the most informative data, thereby reducing the cost of data labeling. Active learning is a machine learning paradigm that can effectively obtain the most informative data from a massive pool of unlabeled data for labeling, and iteratively train the model to improve the accuracy of data labeling. It has been widely used in the field of natural language processing, especially in text classification tasks.

[0094] Since most of the original text in the field of natural language processing is unlabeled, there is only a small amount of labeled text, and the training of text classification models based on deep neural networks depends on a large amount of accurately labeled text, the quality and efficiency of text annotation are important factors to consider in text classification tasks. Active learning can effectively improve the quality and efficiency of text labeling, so it has been widely studied in text classification tasks.

[0095] At present, the methods of labeling text categories based on active learning can be roughly divided into informative selection strategies, representative selection strategies, and hybrid selection strategies. Among them, the informative selection strategy makes predictions based on the uncertainty and inconsistency of the model, which leads to the model being overconfident and insufficiently exploratory, so that the selected text contains repeated information and high label uncertainty, which in turn affects the accuracy of the text classification task; the representative selection strategy relies on the input of the model, such as the vector representation of the text, and the selected text is relatively simple, resulting in a relatively simple category label, which in turn affects the accuracy of the text classification task; and the hybrid selection strategy improves the annotation quality of text category labels by integrating multiple selection strategies, but the model has more parameters, the training is more complex, and the time required is longer.

[0096] In view of this, the embodiments of the present application provide a text tagging method, apparatus, device and storage medium based on active learning. Inspired by the cognitive process of humans to infer and predict through causal information, an interpretable active learning framework for low-resource text classification tasks is provided, which aims to encourage text classification models to prove the rationality of their inferences and conduct in-depth research on unlabeled texts that cannot provide reasonable explanations. The framework uses a pre-trained bidirectional encoder for label prediction and a pre-trained unidirectional decoder to generate explanations of causal relationships in the expression language and evaluate the explanations. Since the explanations expressing causal relationships can help the text classification model accurately understand how to classify text, the text classification model is encouraged to learn to infer the causal relationship between text and labels, reduce the model's dependence on the surface semantics of the text, and deeply explore unlabeled texts that cannot provide reasonable explanations, thereby improving the generalization and accuracy of the model.

[0097] In addition, in order to further enhance the one-way decoder's ability to evaluate explanations, a sorting algorithm for selecting unlabeled texts is designed based on the uncertainty of the bidirectional encoder's predictions and the one-way decoder's evaluation results of the explanations, so as to obtain multiple candidate texts with the most text classification value from the unlabeled texts for annotation, thereby improving the quality of text annotation and reducing the resources occupied by text annotation.

[0098] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0099] like Figure 1 As shown, it is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario diagram includes two terminal devices 110 and a server 120.

[0100] In the embodiment of the present application, the terminal device 110 includes but is not limited to mobile phones, tablet computers, laptop computers, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, car terminals and other devices; a text classification-related client may be installed on the terminal device, which may be software (such as a browser, instant messaging software, etc.), or a web page, a small program, etc. The server 120 is a background server corresponding to the software or web page, small program, etc., or a server dedicated to text classification, which is not specifically limited in the present application. The server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0101] It should be noted that the text labeling method based on active learning in each embodiment of the present application can be executed by an electronic device, which can be a terminal device 110 or a server 120, that is, the method can be executed by the terminal device 110 or the server 120 alone, or by the terminal device 110 and the server 120 together. For example, when executed by the server 120 alone, the server 120 obtains a pre-constructed text sample pool containing a subset of annotated texts and a subset of unannotated texts, and then, based on each predicted text label of a plurality of annotated texts, combined with each pre-annotated real text label, each real explanation and the corresponding evaluation value, calculates the classification loss to adjust the parameters of the text classification model, and based on the model after each parameter adjustment, performs label prediction and evaluation interpretation on the unlabeled text, so as to select a plurality of candidate texts to be labeled from a plurality of unlabeled texts for labeling for the next round of iterative training.

[0102] In the embodiment of the present application, the trained text classification model can be deployed on the server 120 or on the terminal device 110. Then, based on the trained text classification model, the target category to which the target text belongs is determined, and the text processing service is performed based on the determined target category. The text processing service includes but is not limited to text translation, email filtering, voice query, article recommendation, news classification, intelligent question answering, etc.

[0103] In an optional implementation, the terminal device 110 and the server 120 may communicate with each other via a communication network.

[0104] In an optional implementation, the communication network is a wired network or a wireless network.

[0105] It should be noted that Figure 1 The figure is only an example. In fact, the number of terminal devices and servers is not limited and is not specifically limited in the embodiments of the present application.

[0106] In an embodiment of the present application, when there are multiple servers, the multiple servers can form a blockchain, and the servers are nodes on the blockchain; as disclosed in the embodiment of the present application, the text tagging method based on active learning, the text-related data involved can be stored on the blockchain, for example, text samples, text labels, semantic features, text interpretations, evaluation values, etc.

[0107] In addition, the embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving and other scenarios.

[0108] The following describes the text tagging method based on active learning provided by the exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation methods of the present application are not limited in this regard.

[0109] See also Figure 2 , is an architecture diagram of a text classification model based on interpretable active learning, which mainly includes the model training stage and the text selection stage. In the model training stage, the text classification model extracts features from multiple labeled texts selected from the labeled sample subset in the text sample pool, and obtains the predicted text labels of the corresponding labeled texts based on the extracted text features, and calculates the classification loss to adjust the model parameters by combining the pre-annotated real text labels, real explanations and corresponding evaluation values. The classification loss includes the label prediction loss determined by the bidirectional encoder based on the predicted text labels and the real text labels, and the explanation generation loss and explanation ranking loss determined by the unidirectional decoder based on the real explanations and the evaluation values. In the text selection stage, based on the parameter-adjusted text classification model, feature extraction is performed on multiple unlabeled texts selected from the unlabeled sample subset in the text sample pool. The bidirectional encoder determines the information entropy of the corresponding unlabeled text in the label space based on the extracted text features. The unidirectional decoder determines the predicted explanations of the corresponding unlabeled text based on the extracted text features, as well as the conditional logarithmic probability that characterizes the rationality of each predicted explanation. The information entropy and the conditional logarithmic probability are weighted to obtain the ranking index of the corresponding unlabeled text. According to the ranking index, candidate texts with rich text category information are selected from multiple unlabeled texts, and then the selected candidate texts are manually annotated to update the text sample pool. In this way, through multiple rounds of iterative training, a large number of accurate and rich text category labels can be obtained, thereby improving the accuracy of the text classification task.

[0110] In one embodiment, the bidirectional encoder and unidirectional decoder in the text classification model can be constructed using a text-to-text transfer transformer network, such as Figure 3As shown, it is a network architecture diagram of a text classification model provided by an embodiment of the present application, wherein the bidirectional encoder includes N encoding units, each encoding unit adopts a bi-self attention mechanism (Bi-self Attention) and a feed forward network, and the unidirectional decoder includes N decoding units, each decoding unit adopts a causal self attention mechanism (Causal Self Attention), a cross attention mechanism (Cross Attention) and a feed forward network. The bidirectional encoder predicts labels based on the input text, and the decoder generates and evaluates explanations based on the predicted labels and the causal relationship of the input text.

[0111] It should be noted that the embodiments of the present application do not impose any restrictive requirements on the neural network used in the text classification model. For example, a Bert (Bidirectional Encoder Representations from Transformers) network and a GPT (Generative Pre-training Transformer) network may also be used.

[0112] In the text classification task, before marking the text, a text sample pool needs to be pre-built. An optional implementation method is to pre-build the text sample pool in the following way:

[0113] First, collect unstructured natural language corpus in the target business scenario.

[0114] The target business scenarios include but are not limited to at least one of text translation, email filtering, voice query, article recommendation, and news classification. The collected unstructured natural language corpus can come from part or all of web page data, public data sets, book texts (such as novels, essays, and collections), etc.

[0115] Furthermore, after data cleaning of the collected natural language corpus, sentences in the natural language corpus are extracted and used as text samples.

[0116] Taking into account that there may be omissions in the corpus during the collection process, or the data itself is incomplete, and that corpora with large differences may affect category analysis, the data cleaning process of re-examining and verifying the corpus can delete duplicate and useless information, correct existing errors, and provide data consistency.

[0117] Then, the true labels and true explanations are annotated for some text samples to obtain a labeled text subset, and an unlabeled text subset is generated based on the remaining text samples.

[0118] Manually annotate some text samples after data cleaning and mark the real text labels of the corresponding text samples.

[0119] like Figure 4 As shown in the figure, suppose the labeled text is "This apple sells very well" and the labeled true text label is "delicious".

[0120] In order to improve the encoder's ability to evaluate explanations, the annotator annotates multiple real explanations for the marked text by analyzing the causal relationship between the real text label and the corresponding marked text. The multiple real explanations of each marked text include reasonable explanations and unreasonable explanations. Professionals sort the real explanations of each marked text according to the rationality and logic of the annotation specifications to generate an evaluation value for each real explanation.

[0121] In an optional example, the marking result of the marked text is: {marked text: true label; true explanation 1-evaluation value, true explanation 2-evaluation value, ...}.

[0122] like Figure 4 As shown in the figure, the marked text is "This apple sells very well", and the real text label is "delicious". The real explanations that express the causal relationship between the label and the text include: "juicy and sweet", "crisp taste", and "round and regular". Among them, "juicy and sweet" and "crisp taste" can reasonably express the causal relationship between "delicious" and "this apple sells very well". Therefore, the reasonable real explanation is "juicy and sweet" and "crisp taste", and the unreasonable real explanation is "round and regular". The marking result of the marked text is: {This apple sells very well: delicious; juicy and sweet- Crisp taste - Round and straight - }.

[0123] Finally, a text sample pool is generated based on the labeled text subset and the unlabeled text subset.

[0124] Among them, the number of texts in the unlabeled text subset is much larger than the number of texts in the labeled text subset. In this way, the unlabeled text can be automatically labeled through algorithms to reduce the labeling cost and improve the labeling efficiency.

[0125] After building the text sample pool, based on Figure 2 and Figure 3 The text classification model architecture shown in FIG. 1 is a text classification model architecture shown in FIG. 1 , and the present application embodiment provides an implementation process of a text tagging method based on active learning. Figure 5 As shown, taking the server as the execution body as an example, the server performs multiple rounds of iterative training on the text classification model based on a pre-built text sample pool.

[0126] Assuming that the text sample pool is a compact set X, and the label space Y = {1,2,…,M}, the training of the text classification model can be regarded as an M-class classification process on the compact set X and the label space Y, and the text samples are independent and identically distributed (IID) in the space Z = X×Y. The text sampling during the text classification model training can be expressed as {x i ,y i Before text tagging based on active learning, only a small number of text samples in the text sample pool belong to the labeled text subset D l , most of which belong to the unlabeled text subset D u Therefore, we start with the labeled text subset D l Select the labeled text from the dataset for model training, and then select the unlabeled text from the dataset D u Select L candidate texts with the most category information to be annotated and add them to the labeled text subset D l , and has never been marked as this subset D u After multiple rounds of iterations, the text labeling process is completed.

[0127] The implementation process of each iterative training includes the following steps:

[0128] S501: The server performs the following operations respectively for the selected multiple marked texts: using a text classification model to obtain predicted text labels of the marked texts based on the text features of the marked texts.

[0129] In one embodiment, the text classification model has a bidirectional decoder and a unidirectional decoder, and a pre-trained bidirectional encoder is used as a classification backbone network, which can predict labels for multiple input labeled texts and obtain a predicted text label for each labeled text.

[0130] For the specific implementation process, see Figure 6 , mainly includes the following steps:

[0131] S5011: After adding a classification mark to the marked text, it is input into the bidirectional encoder to extract the text features of the marked text, and based on the text features, the context sentence representation of the marked text is obtained.

[0132] In an alternative implementation, the output layer of the bidirectional encoder can be used for token representation, such as sequence tagging, question answering, sentiment analysis, etc., and the [CLS] representation is input to an output layer for classification, so that for each token text x iAfter adding a [CLS] classification mark, it is input into the bidirectional encoder to extract the text features of the marked text, and obtain the context sentence representation of the j token of the corresponding marked text based on each text feature. The formula for the context sentence representation is as follows:

[0133] h j =Encoder([CLS]+x)[j] Formula 1

[0134] Among them, the representation h0 of [CLS] is used as a sentence representation.

[0135] S5012: Based on the context sentence representation, calculate the probability distribution of the marked text in the label space, and based on the probability distribution, obtain the predicted text label of the marked text.

[0136] In an optional implementation, the token representation result h with the [CLS] classification identifier is input into the output layer of the bidirectional encoder (also referred to as the classification layer in the embodiment of the present application), wherein the classification layer is composed of a linear layer and a Softmax function, and the probability distribution of the marked text in the label space can be obtained, and the formula is expressed as follows:

[0137] P(y ic |x i )=Softmax(Linear(h0)) Formula 2

[0138] Among them, c=1,2,…,M.

[0139] Furthermore, from the probability distribution of the marked text in the label space, the label with the highest probability is selected as the predicted text label of the marked text.

[0140] Taking the labeled text "This apple sells very well" as an example, the bidirectional encoder predicts the labels "delicious" and "good-looking". Among them, the probability corresponding to "delicious" is greater than the probability corresponding to "bitter", so "delicious" is used as the predicted text label of the labeled text.

[0141] S502: The server calculates the classification loss based on the obtained predicted text labels, combined with the pre-labeled real text labels, real explanations and corresponding evaluation values.

[0142] In an optional implementation, the classification loss of the text classification model includes the loss generated by the bidirectional encoder and the loss generated by the unidirectional decoder. For the specific calculation process, see Fig. 7A , mainly includes the following steps:

[0143] S5021: Based on each predicted text label and each true text label, obtain the label prediction loss of the text classification model.

[0144] In one example, if Figure 7B As shown in the figure, the real text label is pre-labeled for each marked text, and the predicted text label of each marked text can be obtained through the bidirectional encoder in the text classification model. Therefore, the label prediction loss of the text classification model can be calculated by using the cross entropy loss function to optimize the parameters of the bidirectional encoder, thereby improving the accuracy of label prediction. The calculation formula of the label prediction loss is as follows:

[0145]

[0146] Among them, M is the number of categories in the sample space, y ic is a sign function, taking values ​​of 0 or 1, P(y ic |x i ) is the mark text x i The probability of belonging to category c, N is the number of labeled texts selected in this iteration.

[0147] Since the label prediction loss of the bidirectional encoder is determined based on the information entropy of the labeled text in the label space, the label prediction loss characterizes the uncertainty of the bidirectional encoder label prediction.

[0148] S5022: For multiple marked texts, the following operations are performed respectively: based on the target interpretation selected from each true interpretation corresponding to the marked text and the context sentence representation of the marked text, the sub-interpretation generation loss of the marked text is obtained, and based on the text length of each true interpretation corresponding to the marked text, the sub-interpretation sorting loss of the marked text is obtained.

[0149] In one example, for each marked text {x i ,y ic}, k true explanations including reasonable explanations and unreasonable explanations are pre-labeled {e r} i , i = 0, 1, ..., k-1, and the evaluation value {s r} i , i=0,1,…,k-1Therefore, through each labeled text itself, each true explanation and the corresponding evaluation value, the explanation generation loss of the text classification model can be calculated to optimize the parameters of the unidirectional decoder, thereby improving the accuracy of explanation generation.

[0150] In specific implementation, the calculation process of the sub-explanation generation loss of each labeled text can be found in Figure 8 , mainly includes the following steps:

[0151] S5022_1: Select a target interpretation from each true interpretation according to the evaluation values ​​of each true interpretation of the marked text from large to small, and obtain the target text length of the target interpretation.

[0152] When constructing a text sample pool, multiple real explanations are pre-annotated for each marked text, but these explanations include reasonable explanations and unreasonable explanations, and professionals annotate the real explanations of each marked text with evaluation values ​​in accordance with the rationality and logic of the annotation specifications to sort the real explanations. Therefore, in order to ensure that the one-way decoder generates more standardized explanations, a real explanation with the strongest rationality and logic can be selected from the real explanations of each marked text in order from large to small evaluation values ​​as the target explanation (i.e., the real explanation with the largest evaluation value), and the target text length of the target explanation can be obtained.

[0153] Specifically, the expression formula of the target explanation is:

[0154]

[0155] Here, i is the number of true explanations.

[0156] S5022_2: Based on the target explanation, the context sentence representation of the marked text and the target text length, obtain the sub-explanation generation loss of the marked text.

[0157] like Figure 7B As shown, the context sentence of the labeled text obtained by the bidirectional encoder is represented as h = [h j ] is used as one of the inputs of the unidirectional decoder, and the sub-explanation generation loss of the labeled text is calculated in combination with the target explanation and the target text length. Specifically, the calculation formula for the sub-explanation generation loss of each labeled text is as follows:

[0158]

[0159] in, is the target interpretation, t represents the target text length, Indicates the word before the target text length.

[0160] In one example, considering that each true explanation annotated for the marked text includes a reasonable explanation and an unreasonable explanation, therefore, in order to enable the one-way decoder to distinguish between reasonable explanations and unreasonable explanations, and to sort the reasonable explanations and unreasonable explanations, for the one-way decoder, as shown in the figure Figure 7B As shown, based on each true explanation and the corresponding evaluation value, an explanation ranking loss is designed to optimize the parameters of the unidirectional decoder, so that after training, the unidirectional decoder can rank between reasonable and unreasonable explanations.

[0161] In specific implementation, the calculation process of the sub-explanation ranking loss of each labeled text can be found in Fig. 9 , mainly includes the following steps:

[0162] S5022_3: For each true interpretation of the marked text, the following operations are performed respectively: normalization is performed based on the text length of the true interpretation to obtain the conditional logarithmic probability, which is used to represent the rationality of the true interpretation.

[0163] Specifically, the conditional log probability of each true explanation is calculated as follows:

[0164]

[0165] in, For a true explanation, represents the word before the text length t in the true explanation, p i The conditional log probability normalized for the text length is used to indicate the plausibility of the true explanation.

[0166] S5022_4: Based on the evaluation values ​​of each true interpretation of the marked text, the conditional logarithmic probabilities of each true interpretation are combined in pairs, and the difference between the two conditional logarithmic probabilities in each combination is calculated.

[0167] Assume that the labeled text x i The two predicted explanations are and in, The evaluation value is The evaluation value is and Then The rationality of a and The rationality of b Combine and calculate p a Subtract p b The difference.

[0168] S5022_5: Compare each obtained difference value with a preset value, and determine the sub-explanation sorting loss according to the comparison result.

[0169] Specifically, the default value is 0, and the calculation formula of the sub-explanation ranking loss is as follows:

[0170]

[0171] Among them, p a and p b The calculation formula of is shown in Formula 6.

[0172] S5023: Based on the obtained sub-explanation generation losses, obtain the explanation generation loss of the text classification model.

[0173] Specifically, the sub-explanation generation losses of each labeled text are summed to obtain the explanation generation loss of the unidirectional decoder in the text classification model.

[0174] S5024: Based on the obtained sub-explanation ranking losses, obtain the explanation ranking loss of the text classification model.

[0175] Specifically, the sub-explanation ranking losses of each labeled text are summed to obtain the explanation ranking loss of the unidirectional decoder in the text classification model.

[0176] S5025: Calculate the classification loss based on the label prediction loss, explanation generation loss and explanation ranking loss.

[0177] Specifically, the calculation formula for the overall classification loss of the text classification model is as follows:

[0178] L=L cls +λ1L gen +λ2L rank Formula 8

[0179] Among them, λ1 and λ2 are hyperparameters used to balance the loss weights.

[0180] Since the classification loss includes label prediction loss, explanation generation loss and explanation ranking loss, adjusting the text classification model through label prediction loss can improve the accuracy of label prediction of the bidirectional encoder in the text classification model. Adjusting the text classification model through explanation generation loss and explanation ranking loss can improve the evaluation ability of the unidirectional decoder in the text classification model for explanations, encourage the model to learn to infer the causal relationship between labels and texts, facilitate the text classification model to explore the deep semantics of the text, obtain more accurate text meaning, and improve the trustworthiness of the model, thereby obtaining a better performing model at a lower data cost and mining more text category information for unlabeled texts.

[0181] S503: The server adjusts the parameters of the text classification model based on the classification loss, uses the adjusted text classification model to obtain the predicted text labels and predicted explanations of the selected multiple unlabeled texts, and based on the predicted text labels and the predicted explanations, selects multiple candidate texts to be labeled from the multiple unlabeled texts to update the labeled text subset and the unlabeled text subset.

[0182] In each iterative training process, after adjusting the parameters by the classification loss calculated by the labeled text, in order to u The unlabeled text containing rich text classification information is selected by combining the information entropy predicted by the bidirectional encoder label and the evaluation of the rationality of the explanation by the unidirectional decoder.

[0183] In one example, the unlabeled text selection process is shown in Fig.10 , mainly includes the following steps:

[0184] S5031: For multiple unlabeled texts, the following operations are performed respectively: using the adjusted bidirectional encoder, based on the text features of the unlabeled text, obtaining the context sentence representation of the unlabeled text, and based on the context sentence representation, obtaining the information entropy of the unlabeled text in the label space and normalizing it.

[0185] In the specific implementation, for each unlabeled text, the [CLS] classification mark is added to the unlabeled text and then input into the adjusted bidirectional encoder to obtain the text features of the unlabeled text, and the text features are input into the classification layer of the bidirectional encoder to obtain the context sentence representation of the unlabeled text. Then, based on the context sentence representation of the unlabeled text, the probability distribution of the unlabeled text in the label space is calculated. Since each probability can represent the information entropy of each label prediction in the label space, the larger the probability, the more accurate the prediction. Therefore, the label corresponding to the maximum probability is used as the predicted text label, and the information entropy of the predicted text label is normalized.

[0186] like Fig.11 As shown, assuming that the unlabeled text is "decorating the venue with flowers", through the adjusted bidirectional decoder, the labels obtained include "suitable", "inappropriate", "neutral", etc., among which "suitable" has the largest probability. Therefore, "suitable" is used as the predicted text label of "decorating the venue with flowers".

[0187] S5032: Using a unidirectional decoder, generate at least one predicted interpretation of the unlabeled text based on the predicted text label and context sentence representation of the unlabeled text.

[0188] By inputting the predicted text label and context sentence representation of the unlabeled text obtained by the bidirectional encoder into the unidirectional decoder, at least one predicted explanation can be obtained, wherein each predicted explanation characterizes the causal relationship between the predicted text label and the unlabeled text.

[0189] like Fig.11 As shown, for the unlabeled text "decorate the venue with flowers" and the predicted text label "suitable", the predicted explanations generated by the one-way decoder include "warm and romantic" and "colorful".

[0190] In an optional implementation, a beam search may be used to generate a prediction explanation. The implementation process is shown in Fig.12 , mainly includes the following steps:

[0191] S5032_1: Use the one-way decoder in the text classification model to generate multiple tokens and corresponding word probabilities based on the text features of the unlabeled text.

[0192] As Fig.13A shown, taking the generation of the unlabeled text "Decorate the venue with fresh flowers" as an example, the one-way decoder generates eight tokens: warm, fragrant, romantic, colorful, bright, gorgeous, vivid, fresh, and their corresponding word probabilities.

[0193] S5032_2: Select multiple candidate tokens with a preset beam search number in descending order of their respective word probabilities.

[0194] As Fig.13A shown, assuming the preset beam search number is 2, sort the eight tokens warm, fragrant, romantic, colorful, bright, gorgeous, vivid, fresh in descending order of their respective word probabilities, and select the two candidate tokens "warm" and "colorful" with the highest word probabilities.

[0195] S5032_3: Based on multiple candidate tokens, obtain the phrase probability distribution of the next token sequence, and select multiple candidate token sequences with a preset beam search number in descending order of their respective phrase probabilities.

[0196] As Fig.13A shown, for the candidate token "warm", combine it with the eight tokens generated in the second round of text to obtain multiple candidate token sequences such as "warm warm", "warm fragrant", "warm colorful", "warm vivid", etc., and the phrase probabilities of each candidate token sequence. Sort the multiple candidate token sequences in descending order of their respective phrase probabilities, and select the candidate token sequence "warm fragrant" with the highest phrase probability; similarly, for the candidate token "colorful", combine it with the multiple tokens generated in the second round of text to obtain multiple candidate token sequences such as "colorful acid", "colorful", "colorful gorgeous", "colorful refreshing", etc., and the phrase probabilities of each candidate token sequence. Sort the four token sequences in descending order of their respective phrase probabilities, and select the candidate token sequence "colorful" with the highest phrase probability.

[0197] S5032_4: Determine whether the preset search end condition is met. If so, obtain at least one predicted interpretation; otherwise, return to S5032_3 to repeat the word supplement process for each candidate token sequence.

[0198] As Fig.13AAs shown, if the word-element sequences "warmth" and "color" do not meet the preset search end conditions, the two are respectively combined with multiple word-elements generated by the third round of text to obtain multiple candidate word-element sequences such as "warmth and waves", "warmth and color", "warmth and romance", "warmth and bright", "colorful and bright", "colorful and warm", and "colorful and romance", and the word-element sequences of "warmth and waves" and "colorful and bright" with the largest phrase probability are selected respectively; if these candidate word-element sequences do not meet the preset search end conditions, the two are respectively combined with multiple word-elements generated by the fourth round of text to obtain "warmth and romance" and "colorful and bright" and end the cluster search, so as to obtain the predicted interpretation of "decorating the venue with flowers" as "warmth and romance" and "colorful and bright".

[0199] Optionally, the preset search end conditions include but are not limited to the text length reaching a threshold, encountering an end character, etc.

[0200] S5033: Based on the text length of at least one predicted interpretation, obtain a corresponding conditional logarithmic probability.

[0201] The calculation of the conditional logarithmic probability of the prediction explanation is shown in Formula 6, which is used to indicate the rationality of the prediction explanation.

[0202] S5034: Select a candidate explanation from at least one predicted explanation in the order of the obtained conditional logarithmic probabilities from large to small, and normalize the conditional logarithmic probability of the candidate explanation.

[0203] Specifically, from at least one prediction explanation, the prediction explanation with the largest conditional log probability is selected as a candidate explanation and normalized.

[0204] Taking the unlabeled text "decorating the venue with flowers" as an example, the conditional log probability corresponding to "warm and romantic" is greater than the conditional log probability corresponding to "bright colors", so "warm and romantic" is taken as a candidate explanation for "decorating the venue with flowers".

[0205] S5035: Weight the normalized information entropy and conditional log probability to obtain a ranking index for the unlabeled text.

[0206] Specifically, the formula of the sorting index is as follows:

[0207]

[0208] Among them, c i For unlabeled text x i The information entropy of category c (i.e., predicted text label) in the sample space, p i For unlabeled text x i The conditional log probability corresponding to the candidate explanation of , and λ is a hyperparameter that balances the conditional log probability and information entropy.

[0209] S5036: Select multiple candidate texts to be marked from multiple unmarked texts in descending order of the sorting indicators.

[0210] In one example, by sorting the ranking indicators of multiple unlabeled texts, the first L candidate labeled texts with rich text category information can be obtained for manual labeling, and the labeled candidate texts are put into the labeled text subset D l and the candidate text is taken from the unlabeled text subset D u After multiple rounds of iterations, more valuable unlabeled text can be obtained from the massive unlabeled text subset for annotation. In this way, for low-resource text classification tasks, the workload and labor cost of text annotation required for training models are reduced, the time required for model training is increased, and the interpretability of the model is enhanced, so that the prediction of the model itself has a basis, and the subsequent manual annotation judgment is more reliable.

[0211] Specifically, after the candidate text is selected, different annotators will annotate the text labels of the candidate text, and then different annotators will give corresponding explanations for their annotations. Professionals will then count the different annotation results and obtain the standard labels that meet the annotation specifications as the true labels of the candidate text. At the same time, professionals will also sort the different explanations and obtain the most reasonable explanation as the true explanation of the candidate text.

[0212] like Fig. 13B As shown in the figure, assuming that the candidate text is "decorating the venue with flowers", the text labels given by different annotators are "suitable", "support", "okay", and "oppose", and the statistical votes for each label are 17, 46, 9, and 7 respectively. Among them, the "support" label has the highest number of votes, so "support" is used as the true label of "decorating the venue with flowers". At the same time, the explanations of the "suitable", "support", and "okay" labels given by different annotators are "exquisite and beautiful", "economical and affordable", "warm and romantic", "rich atmosphere", and "suitable season", etc., while the explanations of the "oppose" label given by different annotators are "expensive", "time-consuming and laborious", etc. Among them, after the ranking analysis of professionals, "warm and romantic" is the most reasonable, so "warm and romantic" is used as the true explanation of "decorating the venue with flowers".

[0213] In the embodiment of the present application, in order to make the text classification model learn the causal relationship between the label and the text, a kind of interpretable active learning framework is provided by a bidirectional encoder and a unidirectional decoder, and the model is trained for multiple rounds by the labeled text in the text sample pool, and multiple unlabeled texts to be labeled are screened out from the text sample pool based on the model after each round of training. Wherein, in each round of training, the label (including the real text label and the predicted text label) of the labeled text and the real explanation are used to optimize the model parameters. At the same time, in order to improve the evaluation ability of the unidirectional encoder to the explanation, a sorting algorithm for unlabeled text selection is designed, and the model parameters are optimized by the real explanation of the labeled text and the corresponding evaluation value, so that the model can distinguish reasonable and unreasonable explanations and sort, reduce the model's dependence on the surface semantics of the text, and encourage the classifier to prove the rationality of its label prediction, thereby improving the generalization of the model and realizing more effective exploration in the artificial intelligence framework, obtaining text with more category information for annotation, improving the accuracy of text annotation and reducing text annotation resources at the same time.

[0214] In the embodiment of the present application, the trained text classification model is marked as text, which can be applied to various natural language processing tasks, including but not limited to at least one of text translation, email filtering, voice query, article recommendation, and news classification. Specifically, the target text of the target business is obtained; the target category of the target text is obtained using the trained text classification model, and then the target text is processed based on the target category.

[0215] For example, taking the target business as text translation, assuming that the target text is "There is a dog", the text translation model trained based on the text tag subset can analyze the text type of the target text as English, and then translate it into Chinese "There is a dog" based on the text type of "There is a dog".

[0216] It should be noted that any text processing-related task is applicable to the embodiments of the present application and will not be described in detail here.

[0217] In order to verify the superiority of the explainable active learning framework, the embodiment of the present application was tested in 6 different text classification tasks. The total amount of unlabeled text selected in each task was set to 500, and 100 unlabeled texts were selected for annotation in each iteration. The experimental data of each task is as follows: Fig.14As shown. Among them, each task uses 6 different algorithm frameworks Random (Random uniformly selects unlabeled data for annotation), Maximum Entropy (Max-Entropy, ME), Bayesian Active Learning by Disagreement (Bald), BK (Breaking Ties), Least Confidence (LC), Coreset (Coreset treats the data representations in the labeled pool as Clusters), Contrastive Active Learning (CAL), and XAL for comparison. Among them, the XAL implemented in this application adopts an encoder-decoder model with strong text understanding and generation capabilities, specifically the FLAN-T5-Large model. These 6 models are all trained on a single GPU using the Adam optimizer, with the learning rate set to 1e-4 (ie, 1*10^(-4)) and linear scheduling. The performance of these 6 tasks is evaluated based on the Macro-F1 indicator, and the data results of each task are the average of three initial sets and three random seeds (ie, the average of a total of 9 experimental results).

[0218] Specific:

[0219] Fig.14 (a) in the figure represents a natural language inference task. The dataset used is the Recognizing Textual Entailment (RTE) dataset. The purpose of this task is to detect whether the meaning of a text can be inferred from another text.

[0220] Fig.14 (b) in the figure represents the synonym detection task, which uses the Microsoft Research Paraphrase Corpus (MRPC) dataset. This task requires identifying whether each text sequence pair is synonymous.

[0221] Fig.14 (c) in the figure represents the category sentiment classification task, using the MAMS dataset, which aims to identify the sentiment (such as positive / negative / neutral, etc.) of a target object (such as food, employees, etc.) in a given review;

[0222] Fig.14(d) in the figure represents the stance detection task, which uses the COVID-19 dataset. The task aims to identify the stance of a given text on a target (e.g., in favor / against / neutral).

[0223] Fig.14 (e) in the figure indicates (dis)support detection task, using the DEBA dataset, which aims to detect the stance of a comment reply (such as agree / disagree / neutral).

[0224] Fig.14 (f) in the figure represents the relevance classification task, using the CLEF dataset, which aims to detect whether scientific literature is relevant to a given topic.

[0225] from Fig.14 It can be seen that the XAL framework in the embodiments of the present application is always better than other algorithm frameworks, which fully demonstrates the effectiveness of the XAL framework in processing text classification tasks.

[0226] Specifically, in the RTE task, the XAL framework finally obtained a Macro-f1 score of 83.02%, which is 3.58% higher than the second-best BK's Macro-f1 score of 79.44%; in the DEBA task, the XAL framework finally obtained a Macro-f1 score of 59.91%, which is 2.13% higher than the second-best BK's Macro-f1 score of 57.78%; in the COVID-19 task, probably due to the relatively high complexity of the task, the XAL framework did not significantly outperform other algorithm frameworks at the beginning, but when the text count was 300-500, XAL still showed stronger performance, which also reflected the effectiveness of the XAL framework for unlabeled text selection; in the CLEF task, due to the significant imbalance of label distribution (the ratio of relevant to irrelevant is about 1:21), the performance of other algorithm frameworks was obviously unstable. Even in the challenging scenario of data imbalance, the XAL framework achieved excellent performance and more consistent improvements than other algorithm frameworks in the process of unlabeled text selection, which further verified the effectiveness of XAL.

[0227] In addition, in the above 6 datasets, the implementation also performed a zero-shot setting for the ChatGPT model, where the Macro-F1 scores of the ChatGPT model were 77.42%, 72.46%, 61.28%, 66.67%, 48.96% and 46.41% on the RTE, MRPC, MAMS, COVID-19, DEBA and CLEF datasets, respectively. After supervised fine-tuning on less than 500 labeled texts in the XAL framework, the performance on these datasets was significantly better than ChatGPT, which shows that the XAL framework can achieve more efficient performance with low resources and low costs.

[0228] Based on the same inventive concept, an embodiment of the present application also provides a text labeling device based on active learning. The text classification device can implement the above-mentioned text labeling method based on active learning and achieve the same technical effect.

[0229] See also Fig.15 The text labeling device includes a sample construction module 1501, a label prediction module 1502, a loss calculation module 1503, and a text selection module 1504, wherein:

[0230] A sample construction module 1501 is used to perform multiple rounds of iterative training on a text classification model based on a pre-constructed text sample pool; wherein the text sample pool includes a labeled text subset and an unlabeled text subset;

[0231] The label prediction module 1502 is used to perform the following operations for the selected multiple labeled texts during each iteration of training: using the text classification model to obtain the predicted text label of the labeled text based on the text features of the labeled text;

[0232] A loss calculation module 1503 is used to calculate the classification loss based on the obtained predicted text labels, combined with the pre-labeled real text labels, the real explanations and the corresponding evaluation values, wherein each explanation is used to characterize the causal relationship between the label and the text;

[0233] The text selection module 1504 is used to adjust the parameters of the text classification model according to the classification loss, use the adjusted text classification model to obtain the predicted text labels and predicted explanations of the selected multiple unlabeled texts, and based on the predicted text labels and the predicted explanations, screen out multiple candidate texts to be labeled from the multiple unlabeled texts to update the labeled text subset and the unlabeled text subset.

[0234] Optionally, the text classification model includes a bidirectional encoder, and the label prediction module 1502 is specifically used for:

[0235] After adding a classification mark to the marked text, the marked text is input into the bidirectional encoder to extract text features of the marked text, and based on the text features, a context sentence representation of the marked text is obtained;

[0236] Based on the context sentence representation, the probability distribution of the marked text in the label space is calculated, and based on the probability distribution, the predicted text label of the marked text is obtained.

[0237] Optionally, the text classification model includes a bidirectional encoder and a unidirectional decoder, and the loss calculation module 1503 is specifically used for:

[0238] Using the bidirectional encoder, based on the predicted text labels and the true text labels, obtain the label prediction loss of the text classification model;

[0239] For the multiple marked texts, the one-way decoder is used to perform the following operations respectively: based on the target interpretation selected from each true interpretation corresponding to the marked text and the context sentence representation of the marked text, a sub-interpretation generation loss of the marked text is obtained, and based on the evaluation values ​​of each true interpretation corresponding to the marked text, two-by-two combinations are performed to obtain the sub-interpretation ranking loss of the marked text;

[0240] Based on the obtained sub-explanation generation losses, obtaining the explanation generation loss of the text classification model;

[0241] Based on the obtained sub-explanation ranking losses, obtaining the explanation ranking loss of the text classification model;

[0242] A classification loss is calculated based on the label prediction loss, the explanation generation loss, and the explanation ranking loss.

[0243] Optionally, the loss calculation module 1503 is specifically used for:

[0244] According to the evaluation values ​​of the real interpretations of the marked text from large to small, a target interpretation is selected from the real interpretations, and a target text length of the target interpretation is obtained;

[0245] A sub-interpretation generation loss of the marked text is obtained based on the target interpretation, a context sentence representation of the marked text, and the target text length.

[0246] Optionally, each true interpretation of the marked text includes a reasonable interpretation and an unreasonable interpretation, and the loss calculation module 1503 is specifically used to:

[0247] For each true interpretation of the marked text, the following operations are performed respectively: normalization is performed based on the text length of the true interpretation to obtain a conditional logarithmic probability, where the conditional logarithmic probability is used to represent the rationality of the true interpretation;

[0248] Based on the evaluation values ​​of the true interpretations of the marked text, the conditional logarithmic probabilities of the true interpretations are combined in pairs, and the difference between the two conditional logarithmic probabilities in each combination is calculated;

[0249] The obtained differences are compared with preset values ​​respectively, and the sub-interpretation sorting loss is determined according to the comparison result.

[0250] Optionally, the text classification model includes a bidirectional encoder and a unidirectional decoder, and the text selection module 1504 is specifically used for:

[0251] For the multiple unlabeled texts, the following operations are respectively performed: using the adjusted bidirectional decoder, based on the text features of the unlabeled text, obtaining the context sentence representation of the unlabeled text, and based on the context sentence representation, obtaining the information entropy of the unlabeled text in the label space and normalizing it;

[0252] generating, using the unidirectional decoder, at least one predicted interpretation of the unlabeled text based on the predicted text label of the unlabeled text and the context sentence representation;

[0253] Based on the text length of the at least one predicted interpretation, obtaining a corresponding conditional log probability;

[0254] Selecting a candidate explanation from the at least one predicted explanation in descending order of the obtained conditional logarithmic probabilities, and normalizing the conditional logarithmic probability of the candidate explanation;

[0255] Weighting the normalized information entropy and conditional logarithmic probability to obtain a ranking index of the unlabeled text;

[0256] A plurality of candidate texts to be marked are selected from the plurality of unmarked texts in descending order of the sorting indicators.

[0257] Optionally, the text selection module 1504 is specifically used for:

[0258] Using the unidirectional decoder in the text classification model, based on the text features of the unlabeled text, a plurality of word units and corresponding word probabilities are generated;

[0259] Selecting a preset number of candidate word units in descending order of probability of each word;

[0260] Based on the multiple candidate word-grams, a phrase probability distribution of the next word-gram sequence is obtained, and the multiple candidate word-gram sequences of the preset cluster number are selected in descending order of the probability of each phrase,

[0261] The word supplement process of each candidate word-gram sequence is repeated until a stop condition is satisfied, thereby obtaining the at least one predicted interpretation.

[0262] Optionally, the device further includes a business processing module 1505, and after the text classification model training is completed, the business processing module 1505 is used to:

[0263] Get the target text of the target business;

[0264] Using a trained text classification model to obtain a target category of the target text;

[0265] Based on the target category, performing business processing on the target text;

[0266] The target service includes at least one of text translation, email filtering, voice query, article recommendation, and news classification.

[0267] The text labeling device of the embodiment of the present application, for the text classification task in natural language processing, iteratively trains the text classification model based on a text sample pool including a labeled text subset and an unlabeled text subset. In addition to using the labeled text itself, it also uses the pre-labeled real explanation that characterizes the causal relationship between each labeled text and the real text label, so that the model can accurately understand the deep semantic information of the text, reduce the model's dependence on the surface semantic meaning of the text, and improve the generalization and accuracy of label prediction; at the same time, when calculating the classification loss, in addition to using each predicted text label and each real text label, it also uses the pre-labeled real explanation and the corresponding evaluation value. In this way, the text classification model adjusted based on the classification loss can not only optimize the uncertainty information of the label prediction, but also evaluate the explanation of the causal relationship between the label and the text, so as to select multiple candidate texts with richer category information from the unlabeled text subset for labeling, thereby improving the richness of the text category while reducing the resources occupied by the labeling, thereby improving the accuracy of the text classification task.

[0268] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0269] After introducing the text method and apparatus based on active learning according to an exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0270] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0271] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application. In one embodiment, the electronic device may be a server, such as Figure 1 In this embodiment, the structure of the electronic device can be as follows: Fig.16 As shown, it includes a memory 1601 , a communication module 1603 and one or more processors 1602 .

[0272] The memory 1601 is used to store computer programs executed by the processor 1602. The memory 1601 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0273] The memory 1601 may be a volatile memory, such as a random-access memory (RAM); the memory 1601 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1601 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1601 may be a combination of the above memories.

[0274] The processor 1602 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1602 is used to implement the above-mentioned text tagging method based on active learning when calling the computer program stored in the memory 1601 .

[0275] The communication module 1603 is used to communicate with terminal devices and other servers.

[0276] The specific connection medium between the memory 1601, the communication module 1603 and the processor 1602 is not limited in the embodiment of the present application. Fig.16 In the embodiment, the memory 1601 and the processor 1602 are connected via a bus 1604. The bus 1604 is connected to the processor 1602 via a bus 1604. Fig.16 The connections between the other components are only for illustration and are not intended to be limiting. The bus 1604 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.16 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0277] The memory 1601 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the text tagging method based on active learning in the embodiment of the present application. The processor 1602 is used to execute the above-mentioned text tagging method based on active learning, such as Figure 5 shown.

[0278] In another embodiment, the electronic device may also be other electronic devices, such as Figure 1 The terminal device 110 shown in FIG. 1 is a terminal device 110 shown in FIG. 1 . In this embodiment, the structure of the electronic device can be as follows: Fig.17 As shown, it includes: a communication component 1710, a memory 1720, a display unit 1730, a camera 1740, a sensor 1750, an audio circuit 1760, a Bluetooth module 1770, a processor 1780 and other components.

[0279] The communication component 1710 is used to communicate with the server. In some embodiments, a wireless fidelity (WiFi) module may be included. The WiFi module belongs to a short-range wireless transmission technology. The electronic device can help the user to send and receive information through the WiFi module.

[0280] The memory 1720 can be used to store software programs and data. The processor 1780 executes various functions and data processing of the terminal device 110 by running the software programs or data stored in the memory 1720. The memory 1720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 1720 stores an operating system that enables the terminal device 110 to run. In the present application, the memory 1720 can store an operating system and various application programs, and can also store a computer program for executing the text tagging method based on active learning in the embodiment of the present application.

[0281] The display unit 1730 may also be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the terminal device 110. Specifically, the display unit 1730 may include a display screen 1732 disposed on the front of the terminal device 110. The display screen 1732 may be configured in the form of a liquid crystal display, a light emitting diode, etc. The display unit 1730 may be used to display the browser interface in the embodiment of the present application, etc.

[0282] The display unit 1730 can also be used to receive input digital or character information and generate signal input related to user settings and function control of the terminal device 110. Specifically, the display unit 1730 may include a touch screen 1731 arranged on the front of the terminal device 110, which can collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc.

[0283] The touch screen 1731 can be covered on the display screen 1732, or the touch screen 1731 and the display screen 1732 can be integrated to realize the input and output functions of the terminal device 110, and the integrated display screen can be referred to as a touch display screen. In this application, the display unit 1730 can display the application and the corresponding operation steps.

[0284] Camera 1740 can be used to capture static images, and users can publish images taken by camera 1740 through the application. Camera 1740 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the processor 1780 to convert it into a digital image signal.

[0285] The terminal device may further include at least one sensor 1750, such as an acceleration sensor 1751, a distance sensor 1752, a fingerprint sensor 1753, and a temperature sensor 1754. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0286] The audio circuit 1760, the speaker 1761, and the microphone 1762 can provide an audio interface between the user and the terminal device 110. The audio circuit 1760 can transmit the electrical signal converted from the received audio data to the speaker 1761, which is converted into a sound signal for output. The terminal device 110 can also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 1762 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1760 and converted into audio data, and then the audio data is output to the communication component 1710 to be sent to, for example, another terminal device 110, or the audio data is output to the memory 1720 for further processing.

[0287] The Bluetooth module 1770 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 1770 to exchange data.

[0288] The processor 1780 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device and processes data by running or executing software programs stored in the memory 1720 and calling data stored in the memory 1720. In some embodiments, the processor 1780 may include one or more processing units; the processor 1780 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 1780. In the present application, the processor 1780 can run the operating system, application programs, user interface display and touch response, as well as the text marking method based on active learning in the embodiment of the present application. In addition, the processor 1780 is coupled to the display unit 1730.

[0289] In some possible implementations, various aspects of the text tagging method based on active learning provided by the present application may also be implemented in the form of a program product, which includes a computer program. When the program product is run on an electronic device, the computer program is used to enable the electronic device to perform the steps of the text tagging method based on active learning according to various exemplary embodiments of the present application described above in this specification. For example, the electronic device may perform the following steps: Figure 5 Follow the steps shown in .

[0290] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0291] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, apparatus, or device.

[0292] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0293] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0294] The computer program for performing the operation of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "C" language or similar programming languages. The computer program can be executed entirely on the user electronic device, partially on the user electronic device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In the case of a remote electronic device, the remote electronic device can be connected to the user electronic device through any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external electronic device (for example, using an Internet service provider to connect through the Internet).

[0295] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0296] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0297] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain a computer-usable computer program.

[0298] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program commands. These computer program commands can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the commands executed by the processor of the computer or other programmable data processing device generate commands for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0299] These computer program commands may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the commands stored in the computer readable memory produce an article of manufacture comprising a command device, the command device implementing the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0300] These computer program commands can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the commands executed on the computer or other programmable device provide the instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0301] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0302] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A text tagging method based on active learning, characterized in that: The method comprises: A text classification model is trained for multiple rounds based on a pre-built text sample pool; wherein the text sample pool includes a labeled text subset and an unlabeled text subset, and each iterative training includes: For the selected multiple marked texts, the following operations are performed respectively: using the text classification model, based on the text features of the marked texts, obtaining predicted text labels of the marked texts; Based on the obtained predicted text labels, the classification loss is calculated by combining the pre-annotated real text labels, real explanations and corresponding evaluation values, where each explanation is used to characterize the causal relationship between the label and the text; The parameters of the text classification model are adjusted according to the classification loss, and the adjusted text classification model is used to obtain the predicted text labels and predicted explanations of the selected multiple unlabeled texts respectively, and based on the predicted text labels and the predicted explanations, a plurality of candidate texts to be labeled are screened out from the multiple unlabeled texts to update the labeled text subset and the unlabeled text subset.

2. The method according to claim 1, characterized in that The text classification model includes a bidirectional encoder, and the method of using the text classification model to obtain a predicted text label of the marked text based on the text features of the marked text includes: After adding a classification mark to the marked text, the marked text is input into the bidirectional encoder to extract text features of the marked text, and based on the text features, a context sentence representation of the marked text is obtained; Based on the context sentence representation, the probability distribution of the marked text in the label space is calculated, and based on the probability distribution, the predicted text label of the marked text is obtained.

3. The method according to claim 1, characterized in that The text classification model includes a bidirectional encoder and a unidirectional decoder. The classification loss is calculated based on the obtained predicted text labels, combined with the pre-labeled real text labels, real explanations and corresponding evaluation values, including: Using the bidirectional encoder, based on the predicted text labels and the true text labels, obtain the label prediction loss of the text classification model; For the multiple marked texts, the one-way decoder is used to perform the following operations respectively: based on the target interpretation selected from each true interpretation corresponding to the marked text and the context sentence representation of the marked text, a sub-interpretation generation loss of the marked text is obtained, and based on the evaluation values ​​of each true interpretation corresponding to the marked text, two-by-two combinations are performed to obtain the sub-interpretation ranking loss of the marked text; Based on the obtained sub-explanation generation losses, obtaining the explanation generation loss of the text classification model; Based on the obtained sub-explanation ranking losses, obtaining the explanation ranking loss of the text classification model; A classification loss is calculated based on the label prediction loss, the explanation generation loss, and the explanation ranking loss.

4. The method according to claim 3, characterized in that The step of obtaining a sub-interpretation generation loss of the marked text based on a target interpretation selected from each true interpretation corresponding to the marked text and a context sentence representation of the marked text comprises: According to the evaluation values ​​of the real interpretations of the marked text from large to small, a target interpretation is selected from the real interpretations, and a target text length of the target interpretation is obtained; A sub-interpretation generation loss of the marked text is obtained based on the target interpretation, a context sentence representation of the marked text, and the target text length.

5. The method according to claim 3, characterized in that Each true interpretation of the marked text includes a reasonable interpretation and an unreasonable interpretation, and the evaluation values ​​of each true interpretation corresponding to the marked text are combined in pairs to obtain the sub-interpretation ranking loss of the marked text, including: For each true interpretation of the marked text, the following operations are performed respectively: normalization is performed based on the text length of the true interpretation to obtain a conditional logarithmic probability, where the conditional logarithmic probability is used to represent the rationality of the true interpretation; Based on the evaluation values ​​of the true interpretations of the marked text, the conditional logarithmic probabilities of the true interpretations are combined in pairs, and the difference between the two conditional logarithmic probabilities in each combination is calculated; The obtained differences are compared with preset values ​​respectively, and the sub-interpretation sorting loss is determined according to the comparison result.

6. The method according to any one of claims 1 to 5, characterized in that: The text classification model comprises a bidirectional encoder and a unidirectional decoder. The adjusted text classification model is used to obtain predicted text labels and predicted explanations of the selected multiple unlabeled texts, and multiple candidate texts to be labeled are screened out from the multiple unlabeled texts based on the predicted text labels and the predicted explanations, including: For the multiple unlabeled texts, the following operations are respectively performed: using the adjusted bidirectional decoder, based on the text features of the unlabeled text, obtaining the context sentence representation of the unlabeled text, and based on the context sentence representation, obtaining the information entropy of the unlabeled text in the label space and normalizing it; generating, using the unidirectional decoder, at least one predicted interpretation of the unlabeled text based on the predicted text label of the unlabeled text and the context sentence representation; Based on the text length of the at least one predicted interpretation, obtaining a corresponding conditional log probability; Selecting a candidate explanation from the at least one predicted explanation in descending order of the obtained conditional logarithmic probabilities, and normalizing the conditional logarithmic probability of the candidate explanation; Weighting the normalized information entropy and conditional logarithmic probability to obtain a ranking index of the unlabeled text; A plurality of candidate texts to be marked are selected from the plurality of unmarked texts in descending order of the sorting indicators.

7. The method according to any one of claims 1 to 5, characterized in that The prediction explanation generation process includes: Using the unidirectional decoder in the text classification model, based on the text features of the unlabeled text, a plurality of word units and corresponding word probabilities are generated; Selecting a preset number of candidate word units in descending order of probability of each word; Based on the multiple candidate word-grams, a phrase probability distribution of the next word-gram sequence is obtained, and the multiple candidate word-gram sequences of the preset cluster number are selected in descending order of the probability of each phrase, The word supplement process of each candidate word-gram sequence is repeated until a stop condition is satisfied, thereby obtaining the at least one predicted interpretation.

8. The method according to any one of claims 1 to 5, characterized in that After the text classification model training is completed, the method further includes: Get the target text of the target business; Using a trained text classification model to obtain a target category of the target text; Based on the target category, performing business processing on the target text; The target service includes at least one of text translation, email filtering, voice query, article recommendation, and news classification.

9. A text labeling device based on active learning, characterized in that: include: A sample construction module, used for performing multiple rounds of iterative training on a text classification model based on a pre-constructed text sample pool; wherein the text sample pool includes a labeled text subset and an unlabeled text subset; The label prediction module is used to perform the following operations for the selected multiple labeled texts during each iteration of training: using the text classification model to obtain the predicted text label of the labeled text based on the text features of the labeled text; A loss calculation module, used to calculate the classification loss based on the obtained predicted text labels, combined with the pre-labeled real text labels, the real explanations and the corresponding evaluation values, wherein each explanation is used to characterize the causal relationship between the label and the text; The text selection module is used to adjust the parameters of the text classification model according to the classification loss, and use the adjusted text classification model to obtain the predicted text labels and predicted explanations of the selected multiple unlabeled texts, and based on the predicted text labels and predicted explanations, screen out multiple candidate texts to be labeled from the multiple unlabeled texts to update the labeled text subset and the unlabeled text subset.

10. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any one of the methods of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: It includes a computer program. When the computer program is run on an electronic device, the computer program is used to enable the electronic device to execute the steps of any method described in claims 1 to 8.

12. A computer program product, characterized in that The method comprises a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device executes the steps of any one of the methods described in claims 1 to 8.

Images