Text classification method and device, computer readable storage medium and terminal

Abstract

The invention discloses a text classification method and device, a computer readable storage medium and a terminal. The method comprises the following steps: determining a pre-training text classification model; inputting a to-be-classified text into the pre-trained text classification model to obtain a text classification result; correcting the text classification result at least based on a first category label set; wherein the first category label set is a set of category labels of a training data set of the pre-training text classification model. By adopting the scheme, the accuracy of the obtained text classification result can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of text classification, and in particular, to a text classification method and device, a computer-readable storage medium, and a terminal. Background Art

[0002] Text classification is one of the more common task types in text processing technology. It is widely used in various fields, such as entity recognition, spam filtering, news classification, part-of-speech tagging, etc.

[0003] In the existing related technologies, when implementing text classification, a large number of sample texts are usually annotated first to obtain sample text data and their corresponding text category labels to construct a training data set; then the training data set is input into the model to be trained to obtain a pre-trained text classification model; then the text to be classified is input into the pre-trained text classification model to achieve automatic classification of the text.

[0004] However, due to the limitations of the model itself or deficiencies in the model training method, when the existing technology applies the pre-trained model to the text classification process, the accuracy of the text classification results obtained needs to be improved. Summary of the invention

[0005] The technical problem solved by the embodiments of the present invention is how to improve the accuracy of text classification results.

[0006] To solve the above technical problems, an embodiment of the present invention provides a text classification method, comprising the following steps: a text classification method, characterized in that it includes: determining a pre-trained text classification model; inputting the text to be classified into the pre-trained text classification model to obtain a text classification result; correcting the text classification result based on at least a first category label set; wherein the first category label set is a set of category labels of the training data set of the pre-trained text classification model.

[0007] Optionally, the text classification result is corrected based on at least the first category label set, including: respectively determining the maximum number of consecutive identical characters between the text classification result and each category label in the first category label set, and recording them as the number of identical characters; if the maximum number of identical characters is greater than or equal to a first preset number, then taking the category label to which the maximum number of identical characters belongs as the corrected text classification result.

[0008] Optionally, the method also includes: if the maximum number of identical characters is less than the first preset number, correcting the text classification result based on the similarity between the semantic features of the text classification result and the semantic features of each category label in the second category label set; wherein the second category label set includes the first category label set and its corresponding extended category label set, and each category label in the first category label set is expanded to one or more category labels in the extended category label set.

[0009] Optionally, the first preset number is determined according to the total number of characters included in the text classification result, and the larger the total number of characters is, the larger the first preset number is.

[0010] Optionally, the correction of the text classification result based at least on the first category label set includes: performing word segmentation processing on the text classification result and each category label in the first category label set respectively to obtain a first word segmentation result of the text classification result and a second word segmentation result for each category label; for each second word segmentation result, determining the number of identical word segmentations between the second word segmentation result and the first word segmentation result; determining a ratio of the number of identical word segmentations to the total number of word segmentations in the first word segmentation result as the precision rate, and determining a ratio of the number of identical word segmentations to the total number of word segmentations in the second word segmentation result as the recall rate; determining the F1 score of the category label to which each second word segmentation result belongs according to the precision rate and the recall rate; if the maximum F1 score among the category labels is greater than or equal to a preset score, taking the category label to which the maximum F1 score belongs as the corrected text classification result.

[0011] Optionally, the method also includes: if the maximum F1 score is less than the preset score, correcting the text classification result based on the similarity between the semantic features of the text classification result and the semantic features of each category label in a second category label set; wherein the second category label set includes the first category label set and its corresponding extended category label set, and each category label in the first category label set is expanded to one or more category labels in the extended category label set.

[0012] Optionally, a semantic feature similarity between each category tag in the first category tag set and its corresponding category tag in the extended category tag set is greater than or equal to a preset similarity threshold.

[0013] Optionally, based on the similarity between the semantic features of the text classification result and the semantic features of each category label in the second category label set, the text classification result is corrected, including: determining the semantic features of the text classification result, recorded as first semantic features, and determining the semantic features of each category label in the second category label set, recorded as second semantic features; respectively determining the similarity between the first semantic feature and each second semantic feature; in the second category label set, determining the category label with the greatest similarity to the first semantic feature; and determining the corrected text classification result based on the category label with the greatest similarity.

[0014] Optionally, determining a corrected text classification result based on the category label with the greatest similarity includes: if the category label with the greatest similarity belongs to the first category label set, using the category label with the greatest similarity as the corrected text classification result; if the category label with the greatest similarity belongs to the extended category label set, using the category label corresponding to the category label with the greatest similarity in the first category label set as the corrected text classification result.

[0015] Optionally, determining the pre-trained text classification model includes: determining an initialized text classification model, the initialized text classification model includes an encoding sub-model, a decoding sub-model and a text classification sub-model; determining a training data set, the training data set includes multiple text data and a category label for each text data; using a target loss function, iteratively training one or more of the following items to be optimized to obtain the pre-trained text classification model: the encoding sub-model, the decoding sub-model and the text classification sub-model; wherein, in each round of iteration, the text data in the training data set is input into the encoding sub-model for encoding, and then the encoded data is input into the text classification sub-model for text classification processing, and the encoded data and the category label of the text data are input into the decoding sub-model for decoding.

[0016] Optionally, the target loss function is obtained by weighted operation of a preset first text classification loss function and a second text classification loss function; during the iterative training process, the first text classification loss function is used to iteratively optimize the parameters of the encoding sub-model and the text classification sub-model, and the second text classification loss function is used to iteratively optimize the parameters of the encoding sub-model and the decoding sub-model.

[0017] Optionally, the target loss function is obtained by weighted summing the first text classification loss function and the second text classification loss function.

[0018] Optionally, each piece of text data is obtained by concatenating original text and prompt text; the prompt text is used to indicate the training data set and the target task of the initialized text classification model.

[0019] Optionally, the text classification sub-model is selected from any one of the following: a network structure consisting of multiple linear layers, Dropout layers and activation functions; a model consisting of multiple self-attention mechanism modules; a model consisting of a single linear classification layer.

[0020] An embodiment of the present invention also provides a text classification device, comprising: a model determination module, used to determine a pre-trained text classification model; a text classification module, used to input the text to be classified into the pre-trained text classification model to obtain a text classification result; a text classification result correction module, used to correct the text classification result based on at least a first category label set; wherein the first category label set is a set of category labels of the training data set of the pre-trained text classification model.

[0021] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned text classification method are executed.

[0022] An embodiment of the present invention further provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the steps of the above-mentioned text classification method when running the computer program.

[0023] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0024] In practical applications, the process of training a model using a training data set containing category labels is the process of enabling the trained model to learn the mapping relationship between text and text category. The process of using a pre-trained model for reasoning (the reasoning described in the embodiment of the present invention refers to text classification) is the process of the model determining the text category based on the learned mapping relationship. Therefore, the reasoning result (also called text classification result) of the pre-trained text classification model is actually closely related to the category label set (i.e., the first category label set) of the training data set used in the training process. Normally, the reasoning result of the pre-trained text classification model will have a high similarity with the category labels in the first category label set, and ideally should be completely consistent with one of the category labels in the first category label set.

[0025] Based on the above principles, in an embodiment of the present invention, a set of category labels of the training data set of the pre-trained text classification model (i.e., the first category label set) is used to correct the text classification results output by the pre-trained text classification model. Compared with the prior art that directly uses the model inference results as the final text classification results, this implementation helps to improve the accuracy of the text classification results.

[0026] Further, in an embodiment of the present invention, based on the principle of the aforementioned model training process, ideally the text classification result will be completely consistent with one of the category labels in the first category label set. Based on this, a text classification result correction scheme based on "string matching" is provided. Specifically, the category label with the largest number of identical characters with the text classification result is determined in the first category label set. It can be understood that the more identical characters there are between the category label and the text classification result output by the model, the greater the possibility that the corresponding category label is a true text classification result. Compared with the prior art that directly uses the model reasoning result as the final text classification result, this embodiment uses the category label to which the largest number of identical characters determined after character matching with the first category label set belongs as the corrected text classification result. In this way, it is possible to avoid using inaccurate text classification results caused by model reasoning errors as the final text classification results, thereby improving the accuracy of the text classification results.

[0027] Furthermore, in an embodiment of the present invention, a text classification result correction scheme based on "word segmentation matching (or word matching)" is also provided. Specifically, the category label with the largest F1 score with the text classification result is determined in the first category label set. It can be understood that the larger the F1 score, the greater the possibility that the corresponding category label is a true text classification result. Compared with the prior art that directly uses the model reasoning result as the final text classification result, this implementation scheme uses the category label to which the maximum F1 score determined after word segmentation matching with the first category label set belongs as the corrected text classification result. In this way, it is possible to avoid using inaccurate text classification results caused by model reasoning errors as the final text classification results, thereby improving the accuracy of the text classification results.

[0028] Further, in an embodiment of the present invention, if the maximum number of identical characters is less than the first preset number, it means that the number of consecutive identical characters between the text classification result and each category label of the first category label set is small. Alternatively, if the maximum F1 score in each category label is less than the preset score, it means that the number of identical segmented words between the text classification result and each category label of the first category label set is small. However, since the number of consecutive identical characters or the number of identical segmented words can only represent the size of the difference at the language expression level, in actual applications, for sentences or words with the same or similar semantics, their expressions may not be similar or even completely different (for example, the order of characters or words is inconsistent), resulting in low string matching and segmented word matching, but sentences or words with different expressions may have the same or similar meanings.

[0029] Based on the above reasons, this embodiment further provides a text classification result correction scheme based on semantic feature similarity matching for the case where the text classification result and the string matching degree or word segmentation matching degree of the category label are not high. In this way, the shortcomings of the correction scheme based on "string matching" and the correction scheme based on "word segmentation matching" can be made up to obtain accurate text classification results.

[0030] Furthermore, in an embodiment of the present invention, considering that the same semantics can have a variety of different expressions, therefore, in the process of correcting the text classification results based on "semantic feature similarity", a second category label set is used that is determined by expanding the category labels in the first category label set, that is, the second category label set includes the first category label set and its corresponding extended category label set. In this way, the total number of category labels for semantic feature matching with the text classification results can be expanded, which helps to find category labels with similar semantics, avoid correction errors caused by insufficient number of category labels, and thus improve the accuracy of the corrected text classification results.

[0031] Furthermore, in an embodiment of the present invention, a more optimized pre-trained text classification model is used. Specifically, in the prior art, a model based on Encoder-Decoder (for example, Bart model, T5 model, etc.) is usually used for training to obtain a pre-trained text classification model. This model relies on a supervised text classification method, and the training effect and model reasoning results are not optimized enough. In contrast, this embodiment makes improvements and expansions in the model structure. On the basis of the model architecture based on Encoder-Decoder, a text classification sub-model is added, so that the initialization text classification model used in this embodiment includes both an "Encoder+Decoder" structure and an "Encoder+text classification sub-model" structure, and can perform supervised text classification and generative text classification at the same time. Thus, the optimization of the model training effect is achieved through the improvement of the model structure.

[0032] Furthermore, in an embodiment of the invention, the loss function used in the model training process is also optimized. Specifically, by introducing two loss functions (i.e., the first text classification loss function and the second text classification loss function), and performing a weighted operation on the two loss functions, a target loss function of the training process is obtained, and the two loss functions are respectively used to optimize the parameters of different sub-model combinations in the initialized text classification model.

[0033] From the above, compared with the use of a single loss function for model training, this implementation scheme integrates two parts of loss, which can not only strengthen the understanding of the input text by the encoding sub-model part, but also strengthen the connection between the text data input by the encoding sub-model and the category label input by the decoding sub-model, so that the text classification results generated by the decoding sub-model part are closer to the true category of the text, and reduce the generation of new and irrelevant texts, thereby helping to improve the refinement of model training and obtain a more optimized pre-trained text classification model. Furthermore, based on a more optimized pre-trained text classification model, it helps to improve the accuracy of the text classification results (i.e., the text classification results before correction) obtained by subsequent text classification using the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flow chart of a text classification method in an embodiment of the present invention;

[0035] Figure 2 yes Figure 1 A flowchart of a specific implementation of step S11;

[0036] Figure 3 is a structural diagram of an initialization text classification model in an embodiment of the present invention;

[0037] Figure 4 yes Figure 1 Flow chart of a first specific implementation of step S13;

[0038] Figure 5 yes Figure 1 Flow chart of a second specific implementation of step S13;

[0039] Figure 6 yes Figure 4 A flowchart of a specific implementation of step S44;

[0040] Figure 7 It is a structural schematic diagram of a text classification device in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned objects, features and beneficial effects of the present invention more obvious and understandable, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0042] Reference Figure 1 , Figure 1 1 is a flowchart of a text classification method in an embodiment of the present invention. The method may include steps S11 to S13:

[0043] Step S11: Determine a pre-trained text classification model;

[0044] Step S12: inputting the text to be classified into the pre-trained text classification model to obtain a text classification result;

[0045] Step S13: Correcting the text classification result at least based on a first category label set; wherein the first category label set is a set of category labels of a training data set of the pre-trained text classification model.

[0046] In a specific implementation of step S11, the pre-trained text classification model can adopt an existing trained open source model with text classification function. For example, an encoder-decoder (Encoder-Decoder) based model, including but not limited to a bidirectional and auto-regressive transformer (Bart) model, a T5 model, a Chinese pre-trained unbalanced transformer (CPT) model, etc.

[0047] In another specific implementation of step S11, the pre-trained text classification model can be obtained by training a pre-built initialization text classification model. The initialization text classification model can be a large model including a multi-layer sub-model structure obtained by constructing the encoding sub-model, the decoding sub-model and the text classification sub-model in a preset hierarchical order.

[0048] For the specific structure of the initialization text classification model and its training method, please refer to the following and Figure 2 and Figure 3 Detailed description of the illustrated embodiment.

[0049] Reference Figure 2 , Figure 2 yes Figure 1 The flowchart of a specific implementation of step S11 in the embodiment of the present invention. The step S11 determines the pre-trained text classification model, which may specifically include steps S21 to S23.

[0050] In step S21, an initialization text classification model is determined, where the initialization text classification model includes an encoding sub-model, a decoding sub-model and a text classification sub-model.

[0051] In step S22, a training data set is determined, where the training data set includes a plurality of text data and a category label for each text data.

[0052] In a specific implementation, the category label of each text data can be manually annotated to improve the accuracy of data annotation as much as possible; or, the text data can be input into an existing text classification model to obtain the category label of the input text data to improve the efficiency of annotating the category label as much as possible. Different text data can correspond to the same category label.

[0053] In some embodiments, the category labels may be category labels corresponding to text data in different fields. For example, in the field of entertainment, the category labels may be selected from: music, movies, novels, operas, sketches, games, etc. For another example, in the field of teaching, the category labels may be selected from: Chinese, English, mathematics, history, geography, biology, chemistry, sports, etc.

[0054] In other embodiments, the category labels may also be category labels corresponding to text data of different branches in the same field. For example, in the field of games, text data of different game types (or game branches) may have their own corresponding category labels, which may be selected from: fighting games, makeup (dress-up) games, intellectual games, etc. For another example, in the field of novels, text data of novels from different countries may have their own corresponding category labels, which may be selected from: Chinese novels, British novels, American novels, French novels, etc.

[0055] Furthermore, each piece of text data is obtained by concatenating the original text and the prompt text; the prompt text is used to indicate the training data set and the target task of the initialized text classification model.

[0056] Specifically, in the process of using the training data set to train the model, each text data (i.e., the original text and the prompt text) and its category label can be spliced, and the splicing result can be used as a single input data. Among them, the splicing order of the original text, the prompt text and the category label can be original text + prompt text + category label, or prompt text + original text + category label, or other appropriate splicing order. The embodiment of the present invention does not limit the splicing data of the three in each input data, as long as it can help the optimized text classification model after training to output the expected text classification result.

[0057] In a specific implementation, the prompt text may include prompt words, and there is a many-to-one or one-to-one relationship between the prompt words and the target task. For example, for the target task of "text classification", the corresponding prompt words may be selected from: "text classification", "category", "classification", "label", "text category", "text label".

[0058] In the embodiment of the present invention, the use of prompt words helps to more clearly and accurately indicate the target task of the training data set and the initialized text classification model, thereby ensuring the predictability of the model training effect.

[0059] In a specific embodiment, a certain input data in the training data set may be expressed as follows: "Have Zhang San and Li Si ever participated in a fight? What category is described in this text? Game". The original text is "Have Zhang San and Li Si ever participated in a fight?", the prompt text is "What category is described in this text?" (where the prompt word is "category"), and the category label is "game".

[0060] In another specific embodiment, during the splicing process, the order of the original text and the prompt text can be adjusted. For example, the input data can be expressed as follows: "What is the category of this text? Have Zhang San and Li Si participated in a fight? Game".

[0061] It should be pointed out that the names and contents contained in the above original text are only non-limiting examples. In specific implementations, the original text can be selected or set in combination with actual application scenarios.

[0062] Furthermore, the prompt words contained in the prompt text in each piece of input data, and the expression of the prompt text are not limited to the expression provided in the above embodiment, and the text design and language organization can be appropriately adjusted as long as they do not deviate from the target task that the training model of the training data set should achieve. For example, for the target task of text classification, the corresponding prompt text can also be expressed as "What is the text label of this text?" (where the prompt word is "text label"), "What is the text category of this paragraph" (where the prompt word is "text category"), "What is the text category of this paragraph" (where the prompt word is "text category"), "What is the text category of this paragraph" (where the prompt word is "text category"), "What is the text category of this paragraph" (where the prompt word is "text category"), "What is the text category to which this paragraph belongs" (where the prompt word is "text category"), and so on.

[0063] It is understandable that in practical applications, for some initialization models related to text processing, the target tasks (or functions) that can be achieved may be diverse, including but not limited to text classification, keyword extraction, text summary extraction, automatic question answering, etc. Specifically, the same initialized text processing model can be trained for keyword extraction, trained for text classification, trained for generating text summaries, and so on. Therefore, in the prior art, in the process of text classification using a trained text classification model, it may not be possible to output the expected and accurate text classification results. For example, after the text to be classified is input into a trained text classification model, the output result obtained is a summary or keyword of the text, thereby reducing the user experience.

[0064] To solve the above problems, in an embodiment of the present invention, a prompt text is added on the basis of the original text and its category label to indicate the target task of the training data set and the initialized text classification model. Thus, the model can be trained with the target task indicated by the prompt text as a guide, so that the model training process is more targeted and the training effect meets expectations, which helps to avoid errors such as extracting keywords or generating summaries for the input text to be classified in the subsequent text classification process using the optimized text classification model obtained after training, thereby improving the predictability and accuracy of the text classification results.

[0065] In step S23, the target loss function is used to iteratively train one or more of the following items to be optimized to obtain the pre-trained text classification model: the encoding sub-model, the decoding sub-model and the text classification sub-model.

[0066] In each round of iteration, the text data in the training data set is input into the encoding sub-model for encoding, and then the encoded data is input into the text classification sub-model for text classification processing, and the encoded data and the category label of the text data are input into the decoding sub-model for decoding.

[0067] Combination Figure 3 , Figure 3 It is a structural diagram of an initialization text classification model in an embodiment of the present invention.

[0068] The initialized text classification model may include an encoding sub-model 31, a decoding sub-model 32, and a text classification sub-model 33. The encoding sub-model 31 and the text classification sub-model 33 may constitute a first sub-model combination 313 (or referred to as a first sub-model module 313), and the encoding sub-model 31 and the decoding sub-model 32 may constitute a second sub-model combination 312 (or referred to as a second sub-model module 312).

[0069] As a non-limiting embodiment, the text classification submodel 33 and the decoding submodel 32 are respectively located in the latter layer of the encoding submodel 31. During the model training process, the encoded data output by the encoding submodel 31 is used as input data of the text classification submodel 33 and combined with the category label of the text data as input data of the decoding submodel 32.

[0070] Specifically, the encoding sub-model 31 can encode based on the input data to obtain encoded data (also referred to as encoded hidden state data). For example, the input text data (or binary data obtained by converting the text) can be encoded to obtain a matrix or vector containing several codes, wherein the matrix or vector obtained after encoding generally contains semantic information (or referred to as semantic features) contained in the input text data.

[0071] Specifically, the decoding sub-model 32 can decode based on the input data to obtain decoded data. For example, the text classification result can be output based on the matrix or vector output by the encoding sub-model 31 and the category label of the input text data.

[0072] Specifically, the text classification sub-model 33 can perform text classification based on the input data to obtain a text classification result. For example, the matrix or vector output by the encoding sub-model 31 can be processed to output a text classification result.

[0073] It should be pointed out that since both the decoding sub-model 32 and the text classification sub-model 33 can output text classification results, during the model inference process, one of them can be used as the final text classification result. For example, the output result of the decoding sub-model 32 can be used as the final text classification result.

[0074] Among them, the encoding sub-model 31 can adopt an existing model with data encoding function, for example, it can be a model based on an encoder (Encoder), including but not limited to: Bidirectional Encoder Representations from Transformer (BERT) model, A Robustly Optimized BERT (RoBERTa) model, etc.

[0075] The decoding sub-model 32 can adopt an existing model with data decoding function, for example, a decoder-based model, including but not limited to the second-generation generative pre-trained transformation model (Generative Pre-TrainedTransformer 2, referred to as GPT2), the third-generation generative pre-trained transformation model (Generative Pre-TrainedTransformer 3, referred to as GPT3), the fourth-generation generative pre-trained transformation model (Generative Pre-TrainedTransformer 4, referred to as GPT4), the large language model Bloom, etc.

[0076] It should be pointed out that, in this embodiment, the second sub-model combination 312 composed of the encoding sub-model 31 and the decoding sub-model 32 can adopt the existing encoder-decoder (Encoder-Decoder) based model, including but not limited to the bidirectional and auto-regressive transformation (Bidirectional and Auto-Regressive Transformers, referred to as Bart) model, T5 model, Chinese pre-trained unbalanced transformation (Chinese Pre-trained Unbalanced Transformer, referred to as CPT) model, etc.

[0077] In some embodiments, the text classification submodel 33 may be a network structure consisting of multiple linear layers, Dropout layers (the Dropout layers may be disposed between at least a portion of the linear layers) and activation functions. By introducing multiple linear layers into the text classification submodel 33, it is helpful to map the output data of the encoding submodel 31 to a higher dimensional space, thereby improving the performance of the entire large model. Furthermore, by adding a Dropout layer, the overfitting problem during the training process can be prevented.

[0078] In other embodiments, the text classification sub-model 33 may be a model composed of multiple self-attention modules. By introducing multiple self-attention mechanisms into the text classification sub-model 33, it is possible to perform weighted summation on the output data of the encoding sub-model 31 to learn the important semantic information contained therein, thereby improving the accuracy of text classification.

[0079] In some other embodiments, the text classification sub-model 33 can directly adopt an existing classification network model with text classification function, or adopt a model composed of a single-layer linear classification layer. In this way, the construction process of the text classification sub-model can be omitted, and the construction efficiency of the entire large model can be improved.

[0080] Specifically, the initialized text classification model may further include a tokenizer 34, which includes a class for vectorizing text or converting text into a sequence (ie, decomposing a text stream into tokens, each token being generally a subsequence of characters in the text).

[0081] Furthermore, the target loss function is obtained by weighted operation of a preset first text classification loss function and a second text classification loss function; in the iterative training process, the first text classification loss function is used to iteratively optimize the parameters of the encoding sub-model and the text classification sub-model (i.e., for optimizing Figure 3 The second text classification loss function is used to iteratively optimize the parameters of the encoding sub-model and the decoding sub-model (i.e., to optimize Figure 3 Parameters of the second sub-model combination 312 shown).

[0082] Furthermore, the target loss function is obtained by weighted summing the first text classification loss function and the second text classification loss function.

[0083] In a specific implementation, the target loss function can be expressed by the following expression: Loss = w1 × L classifier +w2×L generate ; w1+w2=1; where Loss represents the target loss function, L classifier represents the first text classification loss function, L generate represents the second text classification loss function, w1 represents the weight of the first text classification loss function, and w2 represents the weight of the second text classification loss function.

[0084] In some non-limiting embodiments, the weight of the first text classification loss function and the weight of the second text classification loss function are the same.

[0085] Specifically, the first text classification loss function is expressed by the following expression:

[0086]

[0087]

[0088] Among them, L classifier represents the first text classification loss function, N represents the total number of text data in the training data set, i represents the sequence number of the text data, L represents the total number of category labels corresponding to the N text data, j represents the sequence number of the category label, y ij represents the coefficient of the i-th text data. When the category label of the i-th text data is the j-th category label, y ij The value of y is 1, otherwise ij The value of p is 0, ij Indicates the probability that the current predicted label of the i-th text data is the j-th category label, It represents the first state matrix output by the text classification sub-model after the i-th text data is input into the encoding sub-model of the initialized text classification model, wherein the category label is the encoded label obtained by encoding the original category label.

[0089] Specifically, the second text classification loss function can be expressed by the following expression:

[0090]

[0091]

[0092] Among them, L generate represents the second text classification loss function, W represents the total number of characters included in the original category label, w represents the character sequence number in the original category label, V represents the total number of characters included in the character set, v represents the character sequence number in the character set, represents the coefficient of the w-th character in the original category label, when the w-th character is the v-th character in the character set, The value of is 1, otherwise The value of p is 0, wv represents the probability that the wth character in the current predicted label is the vth character in the character set, It represents the second state matrix output by the decoding sub-model after the w-th character of the original category label is input into the decoding sub-model of the initialized text classification model.

[0093] In the prior art, a pre-trained text classification model is usually obtained by training based on an Encoder-Decoder model (for example, the Bart model, the T5 model, etc.). This model relies on a supervised text classification method, and the training effect and model inference results are not optimized. In contrast, this embodiment makes improvements and expansions in the model structure (see Figure 3 In the embodiment shown in the figure, a text classification sub-model is added on the basis of the model architecture based on Encoder-Decoder, so that the initialization text classification model adopted in this embodiment includes both the "Encoder+Decoder" structure and the "Encoder+text classification sub-model" structure, and can perform supervised text classification and generative text classification at the same time. Thus, the optimization of the model training effect is achieved through the improvement of the model structure.

[0094] Further, in an embodiment of the invention, the loss function of the training process is also optimized. Specifically, by introducing two parts of loss functions (i.e., the first text classification loss function and the second text classification loss function), and performing weighted operations on the two parts of loss functions, the target loss function of the training process is obtained, and the two parts of loss functions are respectively used to optimize the parameters of different sub-model combinations in the initialized text classification model. Therefore, compared with the use of a single loss function for model training, this implementation scheme integrates two parts of loss, which can not only strengthen the understanding of the input text by the encoding sub-model part, but also strengthen the connection between the encoding sub-model input text data and the category label input by the decoding sub-model, so that the text classification results generated by the decoding sub-model part are closer to the real category of the text, and reduce the generation of new irrelevant texts, thereby helping to improve the refinement of model training and obtain a more optimized pre-trained text classification model. Further, based on a more optimized pre-trained text classification model, it is also helpful to improve the accuracy of the text classification results obtained by subsequent text classification using the model.

[0095] Continue to refer to Figure 1 In the specific implementation of step S12, the text to be classified is input into the pre-trained text classification model to obtain a text classification result.

[0096] It is understandable that if each text data in the training data set used in the model training process is obtained by splicing the original text and the prompt text, then when the pre-trained text classification model is used for model reasoning or application, the text to be classified is also obtained by splicing the original text and the prompt text. In this way, the consistency of the data form of the model training process and the reasoning process can be maintained, and the accuracy of the text classification result can be improved.

[0097] Further, the step S12 (model inference process) may specifically include: encoding the to-be-input text by using the encoding sub-model to obtain encoded data (which may also be referred to as encoded hidden state data); performing at least one round of iteration based on the encoded data, a preset decoding start symbol, and the decoding sub-model to obtain the text classification result. It should be noted that each sub-model involved in the inference process is an optimized sub-model obtained after training.

[0098] Specifically, the iteration process includes: in the first round of iteration, inputting the encoded data and the decoding start symbol into the decoding sub-model to obtain the text category character t of the first round 1 ; in each subsequent round of iteration, inputting the concatenation result of the encoded data, the decoding start symbol, and the text category characters obtained in the previous rounds (encoded data + decoding start symbol + t 1 + t 2 +… t a ) into the decoding sub-model to obtain the text category character of the current round. If the text category character of the current round is not consistent with the preset decoding stop symbol, continue the next round of iteration until the text category character t n obtained in the current round is consistent with the decoding stop symbol; taking the concatenation result of the text category characters obtained in the rounds other than the last round (t 1 + t 2 +… t n-1 ) as the text classification result.

[0099] Among them, the decoding start symbol and the decoding stop symbol can be preset according to the actual situation.

[0100] Combined Figure 3 , the decoding start symbol can be represented by CLS, and the decoding end symbol can be represented by SEP. In the model inference process, in the first round of iteration, inputting the encoded data and the decoding start symbol CLS into the decoding sub-model 32 to obtain the text category character "You" of the first round; in the second round of iteration, inputting the concatenation result of the encoded data, the decoding start symbol CLS, and the character "You" (encoded data + CLS + You) into the decoding sub-model 32 to obtain the text category character "Xi" of the current round; in the third round of iteration, inputting the concatenation result of the encoded data, the decoding start symbol CLS, the character "You", and the character "Xi" (encoded data + CLS + You + Xi) into the decoding sub-model 32 to obtain the text category character "SEP" of the current round. This character is consistent with the decoding stop symbol. At this time, stop the iteration, and take the concatenation result of the text category characters obtained in the rounds other than the last round ("YouXi") as the text classification result.

[0101] In a specific implementation of step S13, the first category label set is a set of category labels of the training data set of the pre-trained text classification model. Specifically, the first category label set is a set of category labels of each text data included in the training data set.

[0102] As a non-limiting embodiment, the category labels of each text data contained in the training data set include the following items: music, movie, novel, drama, sketch, game, then the first category label set is {music, movie, novel, drama, sketch, game}.

[0103] It can be understood that the process of training the initialized text classification model using a training data set containing category labels is the process of enabling the trained model to learn the mapping relationship between text and text category. The process of using a pre-trained model for reasoning (the reasoning described in the embodiment of the present invention refers to text classification) is the process of the model determining the text category based on the learned mapping relationship. Therefore, the reasoning result of the pre-trained text classification model (also called the text classification result) is actually closely related to the category label set of the training data set used in the training process (i.e., the first category label set). Normally, the reasoning result of the pre-trained text classification model will have a high similarity with the category labels in the first category label set, and ideally will be completely consistent with one of the category labels in the first category label set.

[0104] Based on the above principles, in an embodiment of the present invention, a set of category labels of the training data set of the pre-trained text classification model (i.e., the first category label set) is used to correct the text classification results output by the pre-trained text classification model. Compared with the prior art that directly uses the model inference results as the final text classification results, this implementation helps to improve the accuracy of the text classification results.

[0105] Reference Figure 4 , Figure 4 yes Figure 1 The first specific implementation of step S13 may include steps S41 to S44.

[0106] In step S41, the number of consecutive identical characters between the text classification result and the maximum number of each category label in the first category label set is determined respectively, and recorded as the number of identical characters.

[0107] In a specific implementation, a string matching method may be used to determine the number of consecutive identical characters.

[0108] It is understandable that, for each category label in the first category label set, there may be multiple cases where the text classification result and the category label have consecutive identical characters. In this case, the number of consecutive identical characters is determined respectively, and then the maximum value is selected as the number of identical characters. It should be noted that, for the case where there is only one identical character, the value of the number of consecutive identical characters is 1.

[0109] For example, if a category label is "French novel" and the text classification result is "Chinese Novel", there are two cases of consecutive identical characters between the two, namely, "" (space) (the number of consecutive identical characters is 1) and "ovel" (the number of consecutive identical characters is 4). Therefore, for "Frenchnovel" and "Chinese Novel", the maximum number of consecutive identical characters is 4, that is, the number of identical characters is 4.

[0110] For example, if a category label is "battle game" and the text classification result is "game", there is one case where there are two consecutive identical characters between the two, namely, "game". Therefore, for "battle game" and "game", the maximum number of consecutive identical characters is 2, that is, the number of identical characters is 2.

[0111] In step S42, it is determined whether the maximum number of identical characters is greater than or equal to a first preset number.

[0112] The maximum number of identical characters specifically refers to the maximum value of the number of identical characters between the text classification result and each category label in the first category label set.

[0113] Furthermore, the first preset number is determined according to the total number of characters included in the text classification result, and the larger the total number of characters is, the larger the first preset number is. The first preset number is a positive integer.

[0114] Specifically, the first preset number may be a preset ratio of the total number of characters included in the text classification result. The preset ratio may be appropriately set according to the actual application scenario. It is understandable that the higher the accuracy requirement for the text classification result in the actual application scenario, the larger the preset ratio. In a non-limiting manner, the preset ratio may be selected as an appropriate value in the interval [2 / 3, 1].

[0115] For example, if the total number of characters included in the text classification result is 3, the first preset number can be selected from the interval [2, 3]. For another example, if the product of the total number of characters included in the text classification result and the preset ratio is not an integer, the product can be rounded down to determine the value of the first preset number.

[0116] If the judgment result is yes, that is, the maximum number of identical characters is greater than or equal to the first preset number, executing step S43;

[0117] If the judgment result is no, that is, the maximum number of identical characters is less than the first preset number, step S44 is executed.

[0118] In step S43, the category label to which the maximum number of identical characters belongs is used as the corrected text classification result.

[0119] As described above, according to the principle of the model training process, ideally, the text classification result will be completely consistent with one of the category labels in the first category label set. Based on this, in an embodiment of the present invention, a text classification result correction scheme based on "string matching" is provided, and the category label with the largest number of identical characters with the text classification result is determined in the first category label set. It can be understood that the more identical characters there are between the category label and the text classification result output by the model, the greater the possibility that the corresponding category label is a true text classification result.

[0120] From the above, compared with the prior art that directly uses the model inference result as the final text classification result, this embodiment uses the category label to which the largest number of identical characters determined after character matching with the first category label set belongs as the corrected text classification result. In this way, it is possible to avoid using inaccurate text classification results caused by model inference errors as the final text classification results, thereby improving the accuracy of the text classification results.

[0121] In step S44, the text classification result is corrected based on the similarity between the semantic feature of the text classification result and the semantic feature of each category label in the second category label set.

[0122] The second category tag set includes the first category tag set and its corresponding extended category tag set, and each category tag in the first category tag set is extended to one or more category tags in the extended category tag set.

[0123] In a specific implementation, each category tag in the first category tag set is expanded, which specifically means: without changing the semantics (or meaning) of the category tag, the expression of the category tag is appropriately adjusted or converted. For example, if the category tag is "poetry collection", it can be expanded to "poetry collection". For another example, if the category tag is "battle game", it can be expanded to "war game" or "fighting game", "war game", etc.

[0124] It is understandable that when the maximum number of identical characters is less than the first preset number, it means that the number of consecutive identical characters between the text classification result and each category label of the first category label set is small. However, since the number of consecutive identical characters can only represent the difference in language expression, in actual applications, for sentences or words with the same or similar semantics, their expressions may be dissimilar or even completely different (for example, the order of characters or words is inconsistent), resulting in a low string matching degree, but the meanings of sentences or words with different expressions can be the same or close.

[0125] Based on the above reasons, in the embodiment of the present invention, for the case where the text classification result and the string matching degree of the category label are not high, a text classification result correction scheme based on "semantic feature similarity" is further provided. In this way, the shortcomings of the correction scheme based on "string matching" can be compensated and accurate text classification results can be obtained.

[0126] Reference Figure 5 , Figure 5 yes Figure 1 The second specific implementation of step S13 may include steps S51 to S57.

[0127] In step S51, word segmentation processing is performed on the text classification result and each category label in the first category label set to obtain a first word segmentation result of the text classification result and a second word segmentation result of each category label.

[0128] In step S52, for each second word segmentation result, the number of identical word segmentations between the second word segmentation result and the first word segmentation result is determined.

[0129] In step S53, the ratio of the number of identical segmentations to the total number of segmentations in the first segmentation result is determined as the precision, and the ratio of the number of identical segmentations to the total number of segmentations in the second segmentation result is determined as the recall.

[0130] In step S54, the F1 scores of the category labels to which the second word segmentation results belong are determined respectively according to the precision and the recall.

[0131] Specifically, the F1 score (F1 score) may be determined by using the following formula: F1 score=(2×recall rate×precision rate) / (recall rate+precision rate).

[0132] The F1 score can be used to characterize the degree of word matching between the text classification result and the category label in the first category label set. The larger the F1 score, the higher the word matching between the two. The F1 score takes into account both precision and recall, and can be regarded as a harmonic average of the model's precision and recall. Its maximum value is 1 and its minimum value is 0.

[0133] In step S55, it is determined whether the maximum F1 score among the class labels is greater than or equal to a preset score.

[0134] If the judgment result is yes, that is, the maximum F1 score among the category labels is greater than or equal to the preset score, step S56 is executed.

[0135] If the judgment result is no, that is, the maximum F1 score among the category labels is less than the preset score, step S57 is executed.

[0136] In step S56, the category label to which the maximum F1 score belongs is used as the corrected text classification result.

[0137] Specifically, the category label to which the maximum F1 score belongs refers to the category label in the first category label set that has the maximum F1 score between the word segmentation result of the text classification result.

[0138] In a specific implementation, the preset score can be appropriately set according to the actual application scenario. It is understandable that the higher the accuracy requirement for the text classification result, the larger the preset score can be set. In a non-limiting manner, the preset score can be selected from an appropriate value in the interval [0.6, 0.9].

[0139] As described above, according to the principle of the model training process, ideally, the text classification result will be completely consistent with one of the category labels in the first category label set. Based on this, in an embodiment of the present invention, a text classification result correction scheme based on "word segmentation matching (or word matching)" is provided, and the category label with the largest F1 score with the text classification result is determined in the first category label set. It can be understood that the larger the F1 score, the greater the possibility that the corresponding category label is a true text classification result.

[0140] From the above, compared with the prior art that directly uses the model inference result as the final text classification result, this embodiment uses the category label to which the maximum F1 score is determined after word segmentation matching with the first category label set as the corrected text classification result. In this way, it is possible to avoid using inaccurate text classification results caused by model inference errors as the final text classification results, thereby improving the accuracy of the text classification results.

[0141] In step S57, the text classification result is corrected based on the similarity between the semantic feature of the text classification result and the semantic feature of each category label in the second category label set.

[0142] The second category tag set includes the first category tag set and its corresponding extended category tag set, and each category tag in the first category tag set is extended to one or more category tags in the extended category tag set.

[0143] Figure 5 The text classification result correction scheme based on "word segmentation matching" provided in the embodiment shown is similar to Figure 4 The principles of the text classification result correction solutions based on "string matching" provided in the illustrated embodiments are similar, both of which are based on the matching degree at the language expression level, and determine the category label from the first category label set as the text classification result.

[0144] Similarly, when the maximum F1 score among the various category labels is less than the preset score, it means that there are fewer identical segmented words between the text classification result and the various category labels of the first category label set. However, since the number of identical segmented words can only represent the difference in the language expression level, in actual applications, for sentences or words with the same or similar semantics, their expressions may be dissimilar or even completely different (for example, the order of words is inconsistent), resulting in a very low segmented word matching degree, but the meanings of sentences or words with different expressions can be the same or close.

[0145] Based on the above reasons, in the embodiment of the present invention, for the case where the text classification result and the word segmentation of the category label do not match well, a text classification result correction scheme based on "semantic feature similarity" is further provided. In this way, the shortcomings of the correction scheme based on "word segmentation matching" can be compensated to obtain accurate text classification results.

[0146] Reference Figure 6 , Figure 6 yes Figure 4 Flow chart of a specific implementation of step S44 in FIG. The step S44 may specifically include steps S61 to S64.

[0147] In step S61, the semantic feature of the text classification result is determined, recorded as the first semantic feature, and the semantic feature of each category label in the second category label set is determined, recorded as the second semantic feature.

[0148] In step S62, the similarity between the first semantic feature and each second semantic feature is determined respectively.

[0149] In a specific implementation, the method for determining the similarity between semantic features may adopt an existing conventional method, for example, a cosine similarity calculation method.

[0150] In step S63, in the second category label set, a category label having the greatest similarity with the first semantic feature is determined.

[0151] In step S64, a corrected text classification result is determined according to the category label with the greatest similarity.

[0152] Furthermore, if the category label with the greatest similarity belongs to the first category label set, the category label with the greatest similarity is used as the corrected text classification result; if the category label with the greatest similarity belongs to the extended category label set, the category label corresponding to the category label with the greatest similarity in the first category label set is used as the corrected text classification result.

[0153] Specifically, the category label corresponding to the category label with the greatest similarity (i.e., the extended category label) in the first category label set specifically refers to the extended original category label corresponding to the first category label set. For example, if the category label b' with the greatest similarity belongs to the extended category label set B', and b' is obtained by extending the category label b in the first category label set B, then b is used as the corrected text classification result.

[0154] In the embodiment of the present invention, considering that the same semantics can have a variety of different expressions, therefore, in the process of correcting the text classification results based on "semantic feature similarity", a second category label set is used that is determined by expanding the category labels in the first category label set, that is, the second category label set includes the first category label set and its corresponding extended category label set. In this way, the total number of category labels for semantic feature matching with the text classification results can be expanded, which helps to find category labels with similar semantics, avoid correction errors caused by insufficient number of category labels, and thus improve the accuracy of the corrected text classification results.

[0155] It should be pointed out that Figure 5 Step S57 in Figure 4 The content of step S44 in is consistent, so Figure 6The various steps shown are also applicable to step S57.

[0156] Figure 7 : is a schematic diagram of the structure of a text classification device in an embodiment of the present invention. The text classification device may include:

[0157] A model determination module 71, used to determine a pre-trained text classification model;

[0158] A text classification module 72, used to input the text to be classified into the pre-trained text classification model to obtain a text classification result;

[0159] A text classification result correction module 73, configured to correct the text classification result based at least on the first category label set;

[0160] The first category label set is a set of category labels of the training data set of the pre-trained text classification model.

[0161] For the principle, specific implementation and beneficial effects of the text classification device, please refer to the previous article and Figures 1 to 6 The related description about the text classification method shown is not repeated here.

[0162] The embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to execute the above Figures 1 to 6 The steps of the text classification method are shown. The computer-readable storage medium may include a non-volatile memory or a non-transitory memory, and may also include an optical disk, a mechanical hard disk, a solid-state hard disk, and the like.

[0163] Specifically, in the embodiment of the present invention, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0164] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0165] The embodiment of the present invention further provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and the processor executes the above-mentioned Figures 1 to 6 The steps of the text classification method are shown. The terminal may include but is not limited to terminal devices such as mobile phones, computers, tablet computers, etc., and may also be a server, a cloud platform, etc.

[0166] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article indicates that the associated objects before and after are in an "or" relationship.

[0167] The "plurality" appearing in the embodiments of the present application refers to two or more.

[0168] The first, second, etc. descriptions appearing in the embodiments of the present application are only used for illustration and distinction of the description objects. There is no order, nor do they indicate any special limitation on the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0169] It should be noted that the serial numbers of the steps in this embodiment do not limit the execution order of the steps.

[0170] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A text classification method, characterized in that: include: Determine the pre-trained text classification model; Inputting the text to be classified into the pre-trained text classification model to obtain a text classification result; Correcting the text classification result based on at least the first category label set; The first category label set is a set of category labels of the training data set of the pre-trained text classification model.

2. The method according to claim 1, characterized in that: The correcting the text classification result based at least on the first category label set includes: Determine the maximum number of consecutive identical characters between the text classification result and each category label in the first category label set, respectively, and record it as the number of identical characters; If the maximum number of identical characters is greater than or equal to the first preset number, the category label to which the maximum number of identical characters belongs is used as the corrected text classification result.

3. The method according to claim 2, characterized in that The method further comprises: If the maximum number of identical characters is less than the first preset number, correcting the text classification result based on the similarity between the semantic feature of the text classification result and the semantic feature of each category label in the second category label set; The second category tag set includes the first category tag set and its corresponding extended category tag set, and each category tag in the first category tag set is extended to one or more category tags in the extended category tag set.

4. The method according to claim 2 or 3, characterized in that: The first preset number is determined according to the total number of characters included in the text classification result. The larger the total number of characters is, the larger the first preset number is.

5. The method according to claim 1, characterized in that The correcting the text classification result based at least on the first category label set includes: Performing word segmentation processing on the text classification result and each category label in the first category label set respectively to obtain a first word segmentation result of the text classification result and a second word segmentation result of each category label; For each second word segmentation result, determining the number of identical word segmentations between the second word segmentation result and the first word segmentation result; Determine the ratio of the number of identical segmented words to the total number of segmented words in the first segmented word result as the precision rate, and determine the ratio of the number of identical segmented words to the total number of segmented words in the second segmented word result as the recall rate; Determining the F1 scores of the category labels to which the second word segmentation results belong according to the precision and the recall; If the maximum F1 score among the category labels is greater than or equal to the preset score, the category label to which the maximum F1 score belongs is used as the corrected text classification result.

6. The method according to claim 5, characterized in that The method further comprises: If the maximum F1 score is less than the preset score, correcting the text classification result based on the similarity between the semantic feature of the text classification result and the semantic feature of each category label in the second category label set; The second category tag set includes the first category tag set and its corresponding extended category tag set, and each category tag in the first category tag set is extended to one or more category tags in the extended category tag set.

7. The method according to claim 3 or 6, characterized in that: The semantic feature similarity between each category tag in the first category tag set and its corresponding category tag in the extended category tag set is greater than or equal to a preset similarity threshold.

8. The method according to claim 3 or 6, characterized in that: Correcting the text classification result based on the similarity between the semantic feature of the text classification result and the semantic feature of each category label in the second category label set includes: Determine a semantic feature of the text classification result, recorded as a first semantic feature, and determine a semantic feature of each category label in the second category label set, recorded as a second semantic feature; respectively determining a similarity between the first semantic feature and each second semantic feature; Determine, in the second category label set, the category label with the greatest similarity to the first semantic feature; A corrected text classification result is determined according to the category label with the greatest similarity.

9. The method according to claim 8, characterized in that Determining the corrected text classification result according to the category label with the greatest similarity includes: If the category label with the greatest similarity belongs to the first category label set, taking the category label with the greatest similarity as the corrected text classification result; If the category label with the greatest similarity belongs to the extended category label set, the category label corresponding to the category label with the greatest similarity in the first category label set is used as the corrected text classification result.

10. The method according to claim 1, characterized in that The step of determining a pre-trained text classification model comprises: Determine an initialization text classification model, wherein the initialization text classification model includes an encoding sub-model, a decoding sub-model, and a text classification sub-model; Determine a training data set, wherein the training data set includes a plurality of text data and a category label for each text data; Using a target loss function, iteratively training one or more of the following items to be optimized to obtain the pre-trained text classification model: the encoding sub-model, the decoding sub-model, and the text classification sub-model; In each round of iteration, the text data in the training data set is input into the encoding sub-model for encoding, and then the encoded data is input into the text classification sub-model for text classification processing, and the encoded data and the category label of the text data are input into the decoding sub-model for decoding.

11. The method according to claim 10, characterized in that The target loss function is obtained by performing a weighted operation on a preset first text classification loss function and a second text classification loss function; During the iterative training process, the first text classification loss function is used to iteratively optimize the parameters of the encoding sub-model and the text classification sub-model, and the second text classification loss function is used to iteratively optimize the parameters of the encoding sub-model and the decoding sub-model.

12. The method according to claim 11, characterized in that The target loss function is obtained by weighted summing the first text classification loss function and the second text classification loss function.

13. The method according to claim 10, characterized in that Each piece of text data is obtained by splicing the original text and the prompt text; The prompt text is used to indicate the training data set and the target task of the initialized text classification model.

14. The method according to claim 10, characterized in that The text classification sub-model is selected from any of the following: The network structure consists of multiple linear layers, Dropout layers, and activation functions; A model consisting of multiple self-attention mechanism modules; A model consisting of a single linear classification layer.

15. A text classification device, characterized in that: include: A model determination module, used to determine a pre-trained text classification model; A text classification module, used to input the text to be classified into the pre-trained text classification model to obtain a text classification result; A text classification result correction module, used for correcting the text classification result based on at least a first category label set; The first category label set is a set of category labels of the training data set of the pre-trained text classification model.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the text classification method according to any one of claims 1 to 14 are executed.

17. A terminal comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor runs the computer program, the steps of the text classification method according to any one of claims 1 to 14 are performed.

Images