Text classification model training method, text classification method, and related device

Abstract

The application provides a text classification model training method and device and electronic equipment, which are used for improving the training efficiency of the text classification model. The method comprises the following steps: generating a sample training set according to original category samples and new category samples; obtaining category-independent features through a first feature extraction module in a pre-trained classification model, and obtaining category-dependent features through a second feature extraction module in the pre-trained classification model; updating the first feature extraction module in the pre-trained classification model through the output of a category-independent task module and a preset category-independent loss function; updating the second feature extraction module in the pre-trained classification model through the output of a category-dependent task module and a preset category-dependent loss function; and updating a merged classification module contained in the pre-trained classification model according to the updated first feature extraction module and the updated second feature extraction module, so as to generate a text classification model.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a training method, apparatus and electronic device for a text classification model. Background Technology

[0002] Text classification is a common and important task in the field of natural language processing, and it has been widely applied in areas such as intent recognition and sentiment classification. Currently, text classification and similar tasks have achieved good results when combined with pre-trained models. However, training a new text classification model is time-consuming. This is especially true in business scenarios with a large number of categories that are constantly expanding, requiring the continuous addition of data for new categories to combine the old and new category data and retrain the model.

[0003] In related technologies, when retraining a model by combining data from the old category with data from the new category, the model is usually retrained directly based on the combined dataset. Furthermore, the new model cannot fully utilize the prior knowledge of the old model during training, resulting in low training efficiency. Summary of the Invention

[0004] This application provides a training method, apparatus, and electronic device for a text classification model, which improves the training efficiency of the text classification model.

[0005] Firstly, this application provides a method for training a text classification model, comprising the following steps: A training set is generated based on the original category samples corresponding to the original category labels and the new category samples corresponding to the new category labels; The first feature extraction module in the pre-trained classification model obtains category-independent features corresponding to the sample training set, and the second feature extraction module in the pre-trained classification model obtains category-related features corresponding to the sample training set; wherein, the pre-trained classification model is trained based on the original category samples, and the first feature extraction module is used to extract features related to the category label in the sample, and the second feature extraction module is used to extract features unrelated to the category label in the sample; The category-independent features are input into the category-independent task module, and the first feature extraction module in the pre-trained classification model is updated by the output of the category-independent task module and the preset category-independent loss function. The category-related features are input into the category-related task module, and the second feature extraction module in the pre-trained classification model is updated by the output of the category-related task module and the preset category-related loss function. Based on the updated first feature extraction module and the updated second feature extraction module, the merged classification module included in the pre-trained classification model is updated to generate a text classification model; wherein, the merged classification module is used to merge the output results of the updated first feature extraction module and the updated second feature extraction module, and perform text classification prediction based on the merged result.

[0006] Secondly, this application provides a text classification method, including: Obtain the target text to be classified and input the target text into the text classification model; The first feature extraction module in the text classification model obtains the category-independent features of the target text, and the second feature extraction module in the text classification model obtains the category-related features of the target text. The text classification model uses a merging classification module to merge the category-independent features and category-related features of the target text into a text merging feature, and then uses the text merging feature to predict the category label of the target text; wherein, the text classification model is trained using the method described above.

[0007] Thirdly, this application provides a training device for a text classification model, comprising: The generation module is adapted to generate a sample training set based on the original category samples corresponding to the original category labels and the new category samples corresponding to the new category labels; The feature extraction module is adapted to obtain category-independent features corresponding to the sample training set through a first feature extraction module in the pre-trained classification model, and to obtain category-related features corresponding to the sample training set through a second feature extraction module in the pre-trained classification model; wherein the pre-trained classification model is trained based on the original category samples, and the first feature extraction module is used to extract features related to the category label in the samples, and the second feature extraction module is used to extract features unrelated to the category label in the samples; The first update module is adapted to input the class-independent features into the class-independent task module, and update the first feature extraction module in the pre-trained classification model through the output of the class-independent task module and a preset class-independent loss function. The second update module is adapted to input the category-related features into the category-related task module, and update the second feature extraction module in the pre-trained classification model through the output of the category-related task module and the preset category-related loss function. The third update module is adapted to update the merged classification module included in the pre-trained classification model according to the updated first feature extraction module and the updated second feature extraction module to generate a text classification model; wherein, the merged classification module is used to merge the output results of the updated first feature extraction module and the updated second feature extraction module, and perform text classification prediction based on the merged result.

[0008] Fourthly, this application provides a text classification device, comprising: The acquisition module is adapted to acquire the target text to be classified and input the target text into the text classification model. The extraction module is adapted to obtain category-independent features of the target text through the first feature extraction module in the text classification model, and to obtain category-related features of the target text through the second feature extraction module in the text classification model; The prediction module is adapted to merge the category-independent features and category-related features of the target text into a text merging feature through the merging classification module in the text classification model, so as to predict the category label of the target text based on the text merging feature; wherein, the text classification model is trained by the above method.

[0009] Fifthly, this application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the above-described method.

[0010] Sixthly, this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-described method when executed by a processor / processor core.

[0011] In the embodiments provided in this application, firstly, a first feature extraction module included in the pre-trained classification model obtains category-independent features corresponding to the sample training set, and a second feature extraction module included in the pre-trained classification model obtains first category-related features corresponding to the sample training set. Then, the first feature extraction module included in the pre-trained classification model is updated, and the updated first feature extraction module obtains category-independent features corresponding to the sample training set; the second feature extraction module included in the pre-trained classification model is also updated, and the updated second feature extraction module obtains category-related features corresponding to the sample training set. Next, the category-independent features are evaluated using a category-independent task module and a category-independent loss function to update the first feature extraction module; the category-related features are evaluated using a category-related task module and a category-related loss function to update the second feature extraction module. Finally, based on the updated first and second feature extraction modules, the merged classification module included in the pre-trained classification model is updated, thereby generating a text classification model. The merged classification module is used to merge the outputs of the updated first and second feature extraction modules and perform text classification prediction based on the merged result. Therefore, this approach divides the classification model into two parts: class-independent features (also called general features, weakly correlated with the class) and class-related features (strongly correlated with the class, usually high-frequency features under the corresponding classification method). This information decoupling helps the model learn the two types of features separately. Furthermore, the first feature extraction module is updated using a class-independent task module and a class-independent loss function, while the second feature extraction module is updated using a class-related task module and a class-related loss function. This allows for setting matching loss functions for the characteristics of different feature extraction modules. Moreover, merging the classification modules integrates the two types of features, resulting in more accurate classification results. Thus, this approach, through the division of class-independent and class-related features, enables the model to learn the characteristics of both types of features separately. Furthermore, since the pre-trained classification model is trained based on the original class samples, it facilitates the new model (text classification model) learning the prior knowledge from the old model (pre-trained classification model), thereby improving the training efficiency of the new model.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings are provided to further illustrate the present application and form part of the specification. They are used together with the embodiments of the present application to explain the application and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed example embodiments described with reference to the accompanying drawings, in which: Figure 1 A flowchart illustrating a training method for a text classification model provided in one embodiment of this application; Figure 2 A flowchart of a text classification method provided for yet another embodiment of this application; Figure 3a A flowchart illustrating a specific example of the training method for the text classification model of this application is shown. Figure 3b A schematic diagram of the structure of a pre-trained classification model in a specific example is shown; Figure 4 A block diagram of a training device for a text classification model provided in an embodiment of this application; Figure 5 A block diagram of a text classification device provided in an embodiment of this application; Figure 6 This is a block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] To enable those skilled in the art to better understand the technical solutions of this application, exemplary embodiments of this application are described below in conjunction with the accompanying drawings, including various details of the embodiments of this application to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0015] Where there is no conflict, the various embodiments of this application and the features thereof may be combined with each other.

[0016] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0018] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this application, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0019] The training method for the text classification model according to the embodiments of this application can be executed by electronic devices such as terminal devices or servers. The terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Specifically, the method can be implemented by a processor calling a computer program stored in memory.

[0020] In related technologies, when retraining a model by combining data from the old category with data from the new category, the model is usually retrained directly based on the combined dataset. Furthermore, the new model cannot fully utilize the prior knowledge of the old model during training, resulting in low training efficiency. To address these issues, this application proposes a training method for a text classification model. This method divides the classification model into two parts: category-independent features (also called general features, weakly correlated with the category) and category-related features (strongly correlated with the category, typically high-frequency features under the corresponding classification method). This information decoupling helps the model learn the two types of features separately. On the other hand, the first feature extraction module is updated using a category-independent task module and a category-independent loss function, while the second feature extraction module is updated using a category-related task module and a category-related loss function. This allows for setting matching loss functions based on the characteristics of different feature extraction modules. Moreover, merging the classification modules integrates the two types of features, resulting in more accurate classification results. Therefore, this method, by dividing the model into category-independent features and category-related features, enables the model to learn the characteristics of the two types of features respectively. Furthermore, since the pre-trained classification model is trained based on the original category samples, it is easier for the newly trained model (text classification model) to learn the prior knowledge in the old model (pre-trained classification model), thereby helping to improve the training efficiency of the new model.

[0021] Figure 1 A flowchart illustrating a training method for a text classification model, provided as an embodiment of this application. (Refer to...) Figure 1 The method includes: Step S110: Generate a sample training set based on the original category samples corresponding to the original category labels and the new category samples corresponding to the new category labels.

[0022] In this context, original category samples are those corresponding to the original category labels, while newly added category samples are those corresponding to the new category labels added as the business expands. Original and new category labels belong to the same business scenario or project. Typically, there are multiple original category labels, and one or more new category labels, and each new category label is different from any of the original category labels. For example, if the original category labels include: pre-sales service, order consultation, and product introduction, then the new category labels might include: after-sales service, and / or customer follow-up. In short, the new category samples corresponding to the new category labels are derived as the business expands.

[0023] Accordingly, the original category samples and the newly added category samples are mixed to obtain the sample training set. Therefore, the sample training set mentioned in this step is essentially a newly added sample training set, which contains the sum of the original category samples and the newly added category samples.

[0024] Step S120: Obtain category-independent features corresponding to the sample training set through the first feature extraction module in the pre-trained classification model, and obtain category-related features corresponding to the sample training set through the second feature extraction module in the pre-trained classification model.

[0025] The pre-trained classification model is trained based on the original category samples. The pre-trained classification model is trained on the original sample training set, which consists of the original category samples. Therefore, when training the pre-trained classification model, the newly added category labels and their corresponding newly added category samples have not yet appeared.

[0026] The first feature extraction module is used to extract features from the samples that are irrelevant to the class labels, i.e., class-independent features; the second feature extraction module is used to extract features from the samples that are relevant to the class labels, i.e., class-related features. For example, class-independent features include general features that are irrelevant or weakly relevant to the class labels in the classification scenario, while class-related features include scenario-specific features that are strongly relevant to the class labels in the classification scenario.

[0027] Therefore, in this step, the sample training set is input into the first feature extraction module and the second feature extraction module of the pre-trained classification model, respectively. The pre-trained classification model includes at least two parts: the first feature extraction module and the second feature extraction module. The first feature extraction module extracts general features (i.e., features with low correlation to category division and applicable to various business scenarios or projects) from the sample training set that are unrelated to the business scenarios or projects to which the original and newly added category labels belong. The second feature extraction module extracts category-specific features (i.e., features with high correlation to category division and applicable only to specific classification scenarios) from the sample training set that are related to the business scenarios or projects to which the original and newly added category labels belong. Accordingly, the first feature extraction module in the pre-trained classification model obtains category-independent features corresponding to the sample training set; the second feature extraction module in the pre-trained classification model obtains category-related features corresponding to the sample training set.

[0028] Step S130: Input the class-independent features into the class-independent task module, and update the first feature extraction module in the pre-trained classification model using the output of the class-independent task module and the preset class-independent loss function.

[0029] The category-independent task module performs general tasks unrelated to class and evaluates the accuracy of these general tasks using a category-independent loss function. The first feature extraction module is then updated based on the evaluation results. These category-independent general tasks can be flexibly set by those skilled in the art, as long as the task can comprehensively evaluate the model's ability to recognize general features.

[0030] In one optional implementation, the class-independent task module performs the masked language modeling task, and the class-independent loss function is the same as the masked language modeling loss function. Accordingly, when updating the first feature extraction module in the pre-trained classification model using the class-independent task module and the preset class-independent loss function, it is specifically implemented in the following way: First, a masked language modeling (MLM) task is performed on the category-independent features using the category-independent task module, yielding the task execution results. The MLM task uses a mask token to mask tokens in a sequence and prompts the model to fill the mask with an appropriate token, enabling the model to simultaneously consider the context on the right (the token to the right of the mask) and the context on the left (the token to the left of the mask). In this application scenario, target text words are selected from the training sample set based on the first category-independent features. The MLM task is then performed on these target text words (i.e., masking the target text word and predicting based on its context) to obtain the predicted text words corresponding to the target text words. Specifically, words associated with the first category-independent features are selected from the training sample set as target text words. The MLM task is then performed on these selected target text words to obtain the predicted text words corresponding to the target text words.

[0031] Then, by masking the language modeling loss function, a first loss function value corresponding to the task execution result is calculated. If the first loss function value does not meet the first preset convergence condition (the specific convergence condition can be flexibly set according to the type of loss function), the first feature extraction module is updated based on the first loss function value. The operation of obtaining category-independent features corresponding to the sample training set and subsequent operations are repeated based on the updated first feature extraction module until the first loss function value meets the preset convergence condition. Thus, the first loss function value is used to characterize the similarity between the predicted text word and the target text word; the higher the similarity, the stronger the overall recognition ability of the model. Therefore, if the first loss function value does not meet the first preset convergence condition, it indicates that the overall recognition ability of the model needs improvement. Therefore, the first feature extraction module is updated through error backpropagation, and the above operation is repeated based on the updated first feature extraction module until the first loss function value meets the preset convergence condition. In practice, the class-independent loss function value (i.e., the masked language modeling loss function), the predicted text word, and the target text word can be used to calculate the class-independent loss function value (i.e., the first loss function value). The first feature extraction module in the pre-trained classification model is then updated based on this value. The masked language modeling task masks selected target text words and predicts the masked target text words using the first feature extraction module, obtaining the predicted text words corresponding to the target text words. The class-independent loss function is the masked language modeling loss function corresponding to the masked language modeling task. Because the masked language modeling task covers a wide vocabulary, it can examine the classification model's predictive ability for general features. Furthermore, the masked language modeling loss function allows for continuous updating of the parameters in the first feature extraction module, gradually improving its ability to recognize general features. In practice, after each execution of the masked language modeling task, the prediction accuracy of the first feature extraction module in this task is evaluated using the masked language modeling loss function. If not, the masked language modeling task continues iteratively until the prediction accuracy of the first feature extraction module meets the requirements.

[0032] Step S140: Input the category-related features into the category-related task module, and update the second feature extraction module in the pre-trained classification model using the output of the category-related task module and the preset category-related loss function.

[0033] The category-related task module performs category-specific tasks and evaluates the accuracy of these tasks using a category-related loss function. The second feature extraction module is then updated based on the evaluation results. These category-related tasks can be flexibly set by those skilled in the art, as long as the task accurately assesses the model's ability to recognize specific features.

[0034] In one optional implementation, the category-related task module is used to perform the category prediction task (i.e., the category-specific task mentioned above), and the category-related loss function is the same as the category prediction loss function. Accordingly, when updating the second feature extraction module in the pre-trained classification model using the category-related task module and the preset category-related loss function, it is specifically implemented in the following way: First, the category-related task module performs category prediction processing on category-related features to obtain category prediction results. Specifically, the category-related task module is used to perform prediction processing on features associated with category labels. In this embodiment, the category-related task module can be a category classifier, which predicts the category label corresponding to a certain feature, so as to evaluate the model's ability to recognize category-related features based on the comparison between the predicted category label and the actual category label to which the feature belongs. In an optional implementation, the category-related task module is used to output the classification result corresponding to the original category label, that is, the category labels that the category classifier can output are limited to the original category labels and do not include newly added category labels. This approach is mainly to facilitate the model to retain the prior knowledge learned from the original category samples.

[0035] Then, using the category prediction loss function, a second loss function value corresponding to the category prediction result is calculated. If the second loss function value does not meet the second preset convergence condition (the specific convergence condition can be flexibly set according to the type of loss function), the second feature extraction module is updated based on the second loss function value. The operation of obtaining category-related features corresponding to the sample training set and subsequent operations are repeated based on the updated second feature extraction module until the second loss function value meets the preset convergence condition. If the category prediction result is inconsistent with the actual category, it indicates that the model's ability to recognize category-related features is insufficient, and the second feature extraction module needs to be updated through error backpropagation to improve the model's ability to recognize category-related features.

[0036] Therefore, in the above method, the class prediction operation is mainly performed on the samples included in the training set based on the first category-related features. Then, based on the category-related loss function, the class prediction result, and the actual class result, the class-related loss function value (i.e., the second loss function value) is calculated, and the second feature extraction module included in the pre-trained classification model is updated based on the class-related loss function value. Here, the class-related loss function is a classification discrimination loss function used to identify the classification accuracy. Correspondingly, based on the class prediction result of the target sample (i.e., the predicted class) and the actual class result of the target sample (i.e., the labeled actual class), the class-related loss function value of the classification discrimination loss function is calculated, and this value is used to characterize the accuracy of the classification prediction. In specific implementation, after each class prediction operation, the prediction accuracy of the second feature extraction module is evaluated using the classification discrimination loss function to see if it meets the requirements. If not, the class prediction operation continues iteratively until the prediction accuracy of the second feature extraction module meets the requirements.

[0037] Step S150: Update the merged classification module contained in the pre-trained classification model according to the updated first feature extraction module and the updated second feature extraction module to generate a text classification model.

[0038] The pre-trained classification model, in addition to the aforementioned first feature extraction module, second feature extraction module, category-related task module, and category-independent task module, further includes a merging classification module. This merging classification module combines the outputs of the updated first and second feature extraction modules and performs text classification prediction based on the merged result. Because the merging classification module can perform predictions based on a combination of category-related and category-independent features, it can improve prediction accuracy and efficiency.

[0039] Therefore, this approach divides the classification model into two parts: class-independent features (also called general features, weakly correlated with the class) and class-related features (strongly correlated with the class, usually high-frequency features under the corresponding classification method). This information decoupling helps the model learn the two types of features separately. Furthermore, the first feature extraction module is updated using a class-independent task module and a class-independent loss function, while the second feature extraction module is updated using a class-related task module and a class-related loss function. This allows for setting matching loss functions for the characteristics of different feature extraction modules. Moreover, merging the classification modules integrates the two types of features, resulting in more accurate classification results. Thus, this approach, through the division of class-independent and class-related features, enables the model to learn the characteristics of both types of features separately. Furthermore, since the pre-trained classification model is trained based on the original class samples, it facilitates the new model (text classification model) learning the prior knowledge from the old model (pre-trained classification model), thereby improving the training efficiency of the new model.

[0040] Those skilled in the art can also make various modifications and variations to the above embodiments: For example, in one optional implementation, after each update of the first feature extraction module in the pre-trained classification model, the class-independent features output by the updated first feature extraction module are further compared with the class-independent features output by the first feature extraction module before the update. The comparison result determines whether the first feature extraction module needs to be updated again. Specifically, the class-independent features obtained after the i-th update of the first feature extraction module are obtained; using a first mean squared loss function, a first mean squared error is calculated between the class-independent features obtained after the i-th update and the class-independent features obtained before the update. Based on the first mean squared error, it is determined whether to perform the (i+1)-th update of the first feature extraction module. For example, if the first mean squared error is less than a preset first threshold, then the (i+1)-th update of the first feature extraction module is unnecessary. Here, i is a natural number greater than or equal to 1.

[0041] The category-independent features obtained before the update by the first feature extraction module refer to the category-independent features output by the original pre-trained classification model for the training set. Therefore, the category-independent features obtained before the update by the first feature extraction module remain constant values ​​in each subsequent comparison, while the category-independent features obtained after the i-th update by the first feature extraction module change with the value of i. By calculating the first mean square error between the category-independent features obtained after the i-th update and the category-independent features obtained before the update, we can assess whether the updated first feature extraction module possesses a certain recognition ability. Generally, if the first mean square error is less than a preset first threshold, the updated first feature extraction module is considered to retain both a strong learning ability for the original category labels (i.e., retaining the prior knowledge obtained from previous training) and the ability to recognize newly added category samples. The constraint imposed by the first mean square loss function prevents a large difference between the class-independent features obtained after the update and those obtained before the update, ensuring that the difference remains within an acceptable range. This constraint ensures that the updated model retains as much prior knowledge as possible from the old model.

[0042] Similarly, in one optional implementation, after each update of the second feature extraction module in the pre-trained classification model, the category-related features output by the updated second feature extraction module are compared with those output before the update. The comparison result determines whether the second feature extraction module needs to be updated again. Specifically, when updating the second feature extraction module in the pre-trained classification model, firstly, the category-related features obtained after the i-th update of the second feature extraction module are obtained; then, using the second mean squared loss function, the second mean squared error between the category-related features obtained after the i-th update and the category-related features obtained before the update is calculated. The second mean squared error determines whether to perform the (i+1)-th update of the second feature extraction module. For example, if the second mean squared error is less than a preset second threshold, then the (i+1)-th update of the second feature extraction module is unnecessary. Here, i is a natural number greater than or equal to 1.

[0043] The category-related features obtained before the update by the second feature extraction module refer to the category-related features output by the original pre-trained classification model for the sample training set. Therefore, the category-related features obtained before the update by the second feature extraction module remain constant values ​​in each subsequent comparison, while the category-related features obtained after the i-th update by the second feature extraction module change with the value of i. By calculating the second mean square error between the category-related features obtained after the i-th update and the category-related features obtained before the update, we can assess whether the updated second feature extraction module possesses a certain recognition ability. Generally, if the second mean square error is less than a preset second threshold, the updated second feature extraction module is considered to retain both a strong learning ability for the original category labels (i.e., retaining the prior knowledge obtained from previous training) and the ability to recognize newly added category samples. The constraint imposed by the second mean square loss function prevents a large difference between the updated class-related features obtained by the second feature extraction module and the original class-related features, ensuring that the difference remains within an acceptable range. This constraint ensures that the updated model retains as much prior knowledge as possible from the old model.

[0044] Therefore, in the above method, the first and second feature extraction modules, in addition to updating using the class-independent loss function and the class-related loss function, further utilize the first and second mean square loss functions for updating. The synergy of these two loss functions improves the update performance. Since the comparison results can characterize the similarities and differences between the old and new models, it facilitates the optimization of the new model based on the old model (pre-trained classification model), thereby improving the training efficiency of the new model.

[0045] In another optional implementation, to further improve the model update performance, after each update of the first and second feature extraction modules, the merging classification module is further updated to achieve a holistic model update. Accordingly, when updating the merging classification module included in the pre-trained classification model based on the updated first and second feature extraction modules, this is achieved as follows: First, the merging classification module merges the category-independent features obtained after the i-th update and the category-related features obtained after the i-th update, and predicts the text classification based on the resulting merged category features. Then, the merging classification module is updated based on the text classification prediction result and the classification loss function; and the (i+1)-th update is performed on both the first and second feature extraction modules. Thus, this approach further sets a classification loss function to evaluate the classification ability of the merging classification module, and updates the merging classification module through error backpropagation when the function value of the classification loss function does not meet the convergence condition. Furthermore, considering that the classification ability of the merged classification module is closely related to the feature extraction abilities of the first and second feature extraction modules, the first and second feature extraction modules are further updated based on the function value of the classification loss function. Optionally, the category-independent task module and the category-related task module can also be further updated based on the function value of the classification loss function.

[0046] Therefore, in the above method, the first and second feature extraction modules, in addition to updating using class-independent and class-related loss functions, and further updating using the first and second mean square loss functions, also update the entire pre-trained classification model (including the first feature extraction module, second feature extraction module, class-independent task module, class-related task module, and merging classification module) using a classification loss function. The cooperation of these three types of loss functions improves the update effect. In other words, after each update operation in the first and second feature extraction modules, an update operation is performed using the three types of loss functions. The update operations corresponding to the three types of loss functions can be performed simultaneously or in parallel. In summary, the model is updated at three levels using three loss functions, making the updated model more accurate. The merging classification module outputs classification results corresponding to the original and newly added class labels. That is, the number of labels output by the merging classification model expands with the expansion of newly added class labels to flexibly adapt to the updated business scenario.

[0047] In another alternative implementation, considering that each time a new category is added, the original category needs to be mixed with the new category, after several iterations, the amount of data in the training set will become abnormally large, thus increasing training time. To solve this problem, the training set can be generated from the original category samples and the new category samples as follows: First, perform a clustering operation on multiple original category samples to obtain multiple cluster centers corresponding to the original category labels. Then, select original category samples from the multiple original category samples whose distance from the cluster centers is no greater than a preset distance threshold. Finally, generate the training set based on the selected original category samples and the new category samples. Each cluster center is highly correlated with its corresponding category label. Therefore, by selecting original category samples whose distance from the cluster centers is no greater than the preset distance threshold, representative samples can be selected from the original category samples, i.e., samples with significant category discrimination, thus discarding samples with weak category discrimination. Through the above clustering and filtering operation, the amount of data in the training set can be reduced, improving training efficiency.

[0048] Figure 2 A flowchart illustrating a text classification method provided in yet another embodiment of this application. (Refer to...) Figure 2 The method includes: Step S210: Obtain the target text to be classified and input the target text into the text classification model.

[0049] Step S220: Obtain the category-independent features of the target text through the first feature extraction module in the text classification model, and obtain the category-related features of the target text through the second feature extraction module in the text classification model.

[0050] Step S230: The category-independent features and category-related features of the target text are merged into text merge features by the merge classification module in the text classification model, so as to predict the category label of the target text based on the text merge features; wherein, the text classification model is trained by the above method.

[0051] The category-independent features and category-related features mentioned above are obtained through the first feature extraction module and the second feature extraction module included in the text classification model, respectively. For the specific training method of the text classification model, please refer to the previous embodiment, which will not be repeated here.

[0052] For ease of understanding, Figure 3a A flowchart illustrating a specific example is shown. Figure 3b A schematic diagram of the structure of the pre-trained classification model 30 in this example is shown. Figure 3bAs shown, the pre-trained classification model 30 includes: a first feature extraction module 31, a second feature extraction module 32, a category-independent task module 33, a category-related task module 34, and a merging classification module 35.

[0053] Next, using this example as an example, we will introduce in detail the specific implementation details of the training method of the text classification model in this application: In related technologies, conventional text classification is achieved through the following approach: First, the model is trained on the initial batch of labeled old text data (i.e., original category samples). Then, when a new category is discovered in the business scenario, the newly added category data (i.e., new category samples) is labeled. Next, the original category samples and the new category samples are merged together, and the model is retrained. However, the above method mainly trains the model based on all data, relying on manually labeled data. Once a new category is added, the model needs to be retrained on both the old and new categories. As the number of categories and the dataset increases, the training time also increases accordingly. This is time-consuming and inefficient, which is clearly unacceptable for scenarios with rapid business updates. Moreover, the new model in the above method cannot effectively utilize the existing knowledge of the old model, thus leading to even more inefficient training.

[0054] To address the aforementioned issues, this example proposes a training method for a text classification model, which includes the following steps: Step S301: Obtain the original category samples. In this example, the original category samples are also called old task data.

[0055] Step S302: Use k-means clustering to obtain representative samples contained in the oldtask data.

[0056] In practice, firstly, for the data corresponding to category 1 (i.e., task1), the data of task1 is vectorized and then input into a pre-trained classification model (also called the OldBERT model). Then, the OldBERT model outputs the text features of task1, which are then fed into k-means clustering to obtain cluster centers. Based on each cluster center, the samples closest to that center are selected as representative samples for task1. Samples from other categories are also selected in the same way to obtain representative samples for all Oldtask data.

[0057] Step S303: Merge the representative samples in the Oldtask data with the newly added category samples to obtain the sample training set. The newly added category samples are also called the newtask data.

[0058] Step S304: Vectorize all samples in the training set and input the vectorized samples into the pre-trained classification model (i.e., the old BERT model).

[0059] Step S305: Using the first feature extraction module (also called the general feature module) and the second feature extraction module (also called the task-specific feature module) included in the pre-trained classification model, obtain the category-independent features (i.e., general features) and category-related features (i.e., task-specific features) of the sample training set.

[0060] Step S306: Input the class-independent features into the class-independent task module, and update the first feature extraction module in the pre-trained classification model using the output of the class-independent task module and the preset class-independent loss function.

[0061] For example, the category-independent task module can be a linear layer module of the mask language model (MLM), and the category-independent loss function can be the MLM loss function.

[0062] Step S307: Input the category-related features into the category-related task module, and update the second feature extraction module in the pre-trained classification model using the output of the category-related task module and the preset category-related loss function.

[0063] For example, the category-related task module can be a linear layer module for task classification, and the category-related loss function can be the task classification loss function.

[0064] Step S308: Obtain the class-independent features obtained after the first feature extraction module performs the i-th update; calculate the first mean squared error between the class-independent features obtained after the first feature extraction module performs the i-th update and the class-independent features obtained before the update using the first mean squared loss function, and determine whether to perform the (i+1)-th update of the first feature extraction module based on the first mean squared error. For example, input the class-independent features obtained after the first feature extraction module performs the i-th update and the class-independent features obtained before the update into the MSELoss function (i.e., the first mean squared loss function) for optimization, so that the new model retains as much of the general knowledge learned by the old model as possible.

[0065] Step S309: Obtain the category-related features obtained after the second feature extraction module performs the i-th update; calculate the second mean squared error between the category-related features obtained after the second feature extraction module performs the i-th update and the category-related features obtained before the update using the second mean squared loss function, and determine whether to perform the (i+1)-th update of the second feature extraction module based on the second mean squared error. For example, input the category-related features obtained after the second feature extraction module performs the i-th update and the category-related features obtained before the update into MSELoss (i.e., the second mean squared loss function) for optimization, so that the new model retains as much of the task-specific knowledge learned by the old model as possible.

[0066] Step S310: Based on the updated first feature extraction module and the updated second feature extraction module, update the merged classification module included in the pre-trained classification model to generate a text classification model. For example, merge the general feature CLS vector and the task-specific feature CLS vector output by the model for each sample to obtain the combined CLS vector for that sample, and then output the classification result based on the combined CLS vector. Optionally, the classification result and label can also be fed into the cross-entropy loss function (i.e., the classification loss function mentioned above) to further optimize the NewBERT model.

[0067] Accordingly, after the model training is complete, classification prediction is performed in the following way: First, the text to be classified is vectorized to obtain text vectors. Then, these text vectors are fed into a text classification model, which outputs class-independent features and class-related features obtained through the first and second feature extraction modules. The class-independent and class-related feature CLS vectors are then extracted and merged to obtain a combined CLS vector. Finally, the combined CLS vector is passed through a softmax function to obtain the predicted label, thus yielding the predicted classification result.

[0068] In this example, both the first and second feature extraction modules can consist of a linear layer and an activation function layer (Tanh). The main difference between the first and second feature extraction modules is the loss function they employ. The first feature extraction module uses the MLM loss function for optimization, while the second feature extraction module uses a task-specific classification loss function for optimization.

[0069] In summary, this example combines MLM pre-training and knowledge distillation techniques with information deconstruction, enabling the model to retain both general and task-specific information. By combining task-specific and general information to predict the final classification, the model's classification performance is guaranteed. Furthermore, a representative sample selection method based on k-means clustering is proposed, which can quickly and efficiently filter representative samples from old tasks, thus ensuring the model's classification performance and reducing catastrophic forgetting. Specifically, using k-means to select a small number of representative samples and combining them with new samples for model training significantly reduces training time and allows for rapid model updates.

[0070] It is understood that the various method embodiments mentioned above in this application can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this application will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0071] In addition, this application also provides a training device for a text classification model, an electronic device, and a computer-readable storage medium. All of the above can be used to implement the training method of any text classification model provided in this application. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0072] Figure 4 A block diagram of a training apparatus for a text classification model provided in an embodiment of this application is shown.

[0073] Reference Figure 4 This application provides a text classification model training device 40, which includes: The generation module 41 is adapted to generate a sample training set based on the original category samples corresponding to the original category labels and the new category samples corresponding to the new category labels; The feature extraction module 42 is adapted to obtain category-independent features corresponding to the sample training set through a first feature extraction module in the pre-trained classification model, and to obtain category-related features corresponding to the sample training set through a second feature extraction module in the pre-trained classification model; wherein, the pre-trained classification model is trained based on the original category samples, and the first feature extraction module is used to extract features related to the category label in the samples, and the second feature extraction module is used to extract features unrelated to the category label in the samples; The first update module 43 is adapted to input the class-independent features into the class-independent task module, and update the first feature extraction module in the pre-trained classification model through the output of the class-independent task module and the preset class-independent loss function. The second update module 44 is adapted to input the category-related features into the category-related task module, and update the second feature extraction module in the pre-trained classification model through the output of the category-related task module and the preset category-related loss function. The third update module 45 is adapted to update the merged classification module included in the pre-trained classification model according to the updated first feature extraction module and the updated second feature extraction module to generate a text classification model; wherein, the merged classification module is used to merge the output results of the updated first feature extraction module and the updated second feature extraction module, and perform text classification prediction based on the merged result.

[0074] In one optional implementation, the category-independent task module is used to perform a masked language modeling task, and the category-independent loss function is a masked language modeling loss function; The first update module 43 is specifically adapted to: The category-independent task module performs a masked language modeling task on the category-independent features to obtain the task execution results. By masking the language modeling loss function, the first loss function value corresponding to the task execution result is calculated; If the first loss function value does not meet the first preset convergence condition, the first feature extraction module is updated according to the first loss function value, and the step of obtaining the category-independent features corresponding to the sample training set and its subsequent steps are repeated according to the updated first feature extraction module until the first loss function value meets the preset convergence condition.

[0075] In one optional implementation, the category-related task module is used to perform a category prediction task, and the category-related loss function is a category prediction loss function; The second update module 44 is specifically adapted to: The category-related task module performs category prediction processing on the category-related features to obtain category prediction results. The category-related task module is used to output the classification results corresponding to the original category labels. Calculate the second loss function value corresponding to the category prediction result using the category prediction loss function; If the second loss function value does not meet the second preset convergence condition, the second feature extraction module is updated according to the second loss function value, and the step of obtaining the category-related features corresponding to the sample training set and its subsequent steps are repeated according to the updated second feature extraction module until the second loss function value meets the preset convergence condition.

[0076] In one alternative implementation, the first update module 43 is further adapted to: Obtain the category-independent features obtained after the first feature extraction module performs the i-th update; Using a first mean square loss function, calculate the first mean square difference between the class-independent features obtained after the i-th update by the first feature extraction module and the class-independent features obtained before the update by the first feature extraction module. Based on this first mean square difference, determine whether to perform the (i+1)-th update of the first feature extraction module; and / or, The second update module 44 is also adapted to: Obtain the category-related features obtained after the second feature extraction module performs the i-th update; The second mean squared loss function is used to calculate the second mean squared error between the category-related features obtained after the second feature extraction module performs the i-th update and the category-related features obtained before the second feature extraction module updates. Based on the second mean squared error, it is determined whether to perform the (i+1)-th update of the second feature extraction module.

[0077] In one alternative implementation, the third update module 45 is further adapted to: The merging classification module merges the category-independent features obtained after the i-th update and the category-related features obtained after the i-th update, and predicts the text classification based on the obtained category-merged features; Based on the text classification prediction results and the classification loss function, the merging classification module is updated; and the first feature extraction module is updated for the (i+1)th time, and the second feature extraction module is updated for the (i+1)th time. The merging classification module is used to output classification results corresponding to the original category labels and the newly added category labels.

[0078] In one alternative implementation, the generation module 41 is specifically adapted to: Perform clustering operations on samples from multiple original categories to obtain multiple cluster centers corresponding to the original category labels; From the plurality of original category samples, select original category samples whose distance from the cluster center is not greater than a preset distance threshold; The sample training set is generated based on the selected original category samples and the newly added category samples.

[0079] In one optional implementation, the original category label and the newly added category label belong to the same business scenario or the same business project; wherein there are multiple original category labels and at least one newly added category label; and the newly added category label is different from any of the original category labels.

[0080] Figure 5 A block diagram of a text classification device provided in an embodiment of this application is shown.

[0081] Reference Figure 5 This application provides a text classification device 50, which includes: Acquisition module 51 is adapted to acquire the target text to be classified and input the target text into a text classification model; Extraction module 52 is adapted to obtain category-independent features of the target text through the first feature extraction module in the text classification model, and to obtain category-related features of the target text through the second feature extraction module in the text classification model; Prediction module 53 is adapted to merge the category-independent features and category-related features of the target text into a text merging feature through the merging classification module in the text classification model, so as to predict the category label of the target text based on the text merging feature; wherein, the text classification model uses the above-mentioned... Figure 1 The method shown is used for training.

[0082] Figure 6 This is a block diagram of an electronic device provided in an embodiment of this application.

[0083] Reference Figure 6 This application provides an electronic device, which includes: at least one processor 501; at least one memory 502; and one or more I / O interfaces 503 connected between the processor 501 and the memory 502; wherein the memory 502 stores one or more computer programs that can be executed by the at least one processor 501, and the one or more computer programs are executed by the at least one processor 501 to perform the training method of the above-mentioned text classification model.

[0084] This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor / processor core, implements the above-described text classification model training method. The computer-readable storage medium can be volatile or non-volatile.

[0085] This application also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described text classification model training method.

[0086] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0087] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0088] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0089] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing state information from the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0090] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0091] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0092] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0093] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0095] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this application as set forth by the appended claims.

Claims

1. A method for training a text classification model, characterized in that, include: A training set is generated based on the original category samples corresponding to the original category labels and the new category samples corresponding to the new category labels; The first feature extraction module in the pre-trained classification model obtains category-independent features corresponding to the sample training set, and the second feature extraction module in the pre-trained classification model obtains category-related features corresponding to the sample training set; wherein, the pre-trained classification model is trained based on the original category samples, and the first feature extraction module is used to extract features related to the category label in the sample, and the second feature extraction module is used to extract features unrelated to the category label in the sample; The category-independent features are input into the category-independent task module, and the first feature extraction module in the pre-trained classification model is updated by the output of the category-independent task module and the preset category-independent loss function. The category-related features are input into the category-related task module, and the second feature extraction module in the pre-trained classification model is updated by the output of the category-related task module and the preset category-related loss function. Based on the updated first feature extraction module and the updated second feature extraction module, the merged classification module included in the pre-trained classification model is updated to generate a text classification model; wherein, the merged classification module is used to merge the output results of the updated first feature extraction module and the updated second feature extraction module, and perform text classification prediction based on the merged result.

2. The method of claim 1, wherein, The category-independent task module is used to perform masked language modeling tasks, and the category-independent loss function is the masked language modeling loss function; The step of updating the first feature extraction module in the pre-trained classification model using the class-independent task module and the preset class-independent loss function includes: The category-independent task module performs a masked language modeling task on the category-independent features to obtain the task execution results. By masking the language modeling loss function, the first loss function value corresponding to the task execution result is calculated; If the first loss function value does not meet the first preset convergence condition, the first feature extraction module is updated according to the first loss function value, and the step of obtaining the category-independent features corresponding to the sample training set and its subsequent steps are repeated according to the updated first feature extraction module until the first loss function value meets the preset convergence condition.

3. The method of claim 1, wherein, The category-related task module is used to perform category prediction tasks, and the category-related loss function is the category prediction loss function; The step of updating the second feature extraction module in the pre-trained classification model using the category-related task module and the preset category-related loss function includes: The category-related task module performs category prediction processing on the category-related features to obtain category prediction results. The category-related task module is used to output the classification results corresponding to the original category labels. Calculate the second loss function value corresponding to the category prediction result using the category prediction loss function; If the second loss function value does not meet the second preset convergence condition, the second feature extraction module is updated according to the second loss function value, and the step of obtaining the category-related features corresponding to the sample training set and its subsequent steps are repeated according to the updated second feature extraction module until the second loss function value meets the preset convergence condition.

4. The method of claim 1, wherein, The update of the first feature extraction module in the pre-trained classification model also includes: Obtain the category-independent features obtained after the first feature extraction module performs the i-th update; Using a first mean square loss function, calculate the first mean square difference between the class-independent features obtained after the i-th update by the first feature extraction module and the class-independent features obtained before the update by the first feature extraction module. Based on this first mean square difference, determine whether to perform the (i+1)-th update by the first feature extraction module; and / or, The update of the second feature extraction module in the pre-trained classification model also includes: Obtain the category-related features obtained after the second feature extraction module performs the i-th update; The second mean squared loss function is used to calculate the second mean squared error between the category-related features obtained after the second feature extraction module performs the i-th update and the category-related features obtained before the second feature extraction module updates. Based on the second mean squared error, it is determined whether to perform the i+1-th update of the second feature extraction module.

5. The method of claim 4, wherein, The step of updating the merging classification module included in the pre-trained classification model based on the updated first feature extraction module and the updated second feature extraction module includes: The merging classification module merges the category-independent features obtained after the i-th update and the category-related features obtained after the i-th update, and predicts the text classification based on the obtained category-merged features. Based on the text classification prediction results and the classification loss function, the merging classification module is updated; and the first feature extraction module is updated for the (i+1)th time, and the second feature extraction module is updated for the (i+1)th time. The merging classification module is used to output classification results corresponding to the original category labels and the newly added category labels.

6. The method according to any one of claims 1 to 5, characterized in that, The step of generating a sample training set based on the original category samples corresponding to the original category labels and the new category samples corresponding to the new category labels includes: Perform clustering operations on samples from multiple original categories to obtain multiple cluster centers corresponding to the original category labels; From the plurality of original category samples, select original category samples whose distance from the cluster center is not greater than a preset distance threshold; The sample training set is generated based on the selected original category samples and the newly added category samples.

7. The method according to any one of claims 1-5, characterized in that, The original category labels and the newly added category labels belong to the same business scenario or the same business project; there are multiple original category labels and at least one newly added category label; and the newly added category label is different from any of the original category labels.

8. A text classification method, characterized in that, include: Obtain the target text to be classified and input the target text into the text classification model; The first feature extraction module in the text classification model obtains the category-independent features of the target text, and the second feature extraction module in the text classification model obtains the category-related features of the target text. The text classification model merges the category-independent features and category-related features of the target text into a text merging feature, and predicts the category label of the target text based on the text merging feature; wherein the text classification model is trained by the method described in any one of claims 1-7.

9. A training device for a text classification model, characterized in that, include: The generation module is adapted to generate a sample training set based on the original category samples corresponding to the original category labels and the new category samples corresponding to the new category labels; The feature extraction module is adapted to obtain category-independent features corresponding to the sample training set through a first feature extraction module in the pre-trained classification model, and to obtain category-related features corresponding to the sample training set through a second feature extraction module in the pre-trained classification model; wherein the pre-trained classification model is trained based on the original category samples, and the first feature extraction module is used to extract features related to the category label in the samples, and the second feature extraction module is used to extract features unrelated to the category label in the samples; The first update module is adapted to input the class-independent features into the class-independent task module, and update the first feature extraction module in the pre-trained classification model through the output of the class-independent task module and a preset class-independent loss function. The second update module is adapted to input the category-related features into the category-related task module, and update the second feature extraction module in the pre-trained classification model through the output of the category-related task module and the preset category-related loss function. The third update module is adapted to update the merged classification module included in the pre-trained classification model according to the updated first feature extraction module and the updated second feature extraction module to generate a text classification model; wherein, the merged classification module is used to merge the output results of the updated first feature extraction module and the updated second feature extraction module, and perform text classification prediction based on the merged result.

10. A text classification device, characterized in that, include: The acquisition module is adapted to acquire the target text to be classified and input the target text into the text classification model. The extraction module is adapted to obtain category-independent features of the target text through the first feature extraction module in the text classification model, and to obtain category-related features of the target text through the second feature extraction module in the text classification model; The prediction module is adapted to merge the category-independent features and category-related features of the target text into a text merging feature through the merging classification module in the text classification model, so as to predict the category label of the target text based on the text merging feature; wherein, the text classification model is trained by the method described in any one of claims 1-7.

11. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1-8.

Images