Data enhancement method, device and electronic device for text classification

Abstract

The present application relates to a data enhancement method, device and electronic device for text classification, which improves the enhancement effect of text data and achieves a good balance between event processing and text data enhancement effect. The method comprises: obtaining a training data set for text classification; using a text classification model to determine the predicted category label corresponding to each sample in the training data set; based on the true category label and the predicted category label corresponding to each sample, obtaining a confusion matrix representing the classification accuracy of the text classification model; based on the confusion matrix, determining the enhancement probability corresponding to each category label; for each category label in the training data set, according to the enhancement probability corresponding to the category label, determining the sample to be enhanced from the samples whose true category label is the category label, and performing corresponding data enhancement processing on each sample to be enhanced; replacing the sample corresponding to the training data set with the enhanced sample to obtain the enhanced training data set.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a data enhancement method, device and electronic device for text classification. Background Art

[0002] In solving various tasks in natural language processing, data augmentation of original text has been proven to be effective, and the combination strategy of multiple data augmentation methods can further improve the performance of the model. When enhancing data, the general practice is to apply a certain enhancement method or a certain combination of enhancement strategies to all text data indiscriminately, and the parameters used in the enhancement method are usually specified based on human experience, or obtained through repeated training of the model and reverse iteration, such as the probability of selecting a certain enhancement method in the combination of enhancement strategies. However, human experience specification lacks reasonable guidance and is not rational enough. The practice of repeatedly training the model to find the optimal parameters will increase the search space and computational cost as the number of parameters increases, making it difficult to balance the computational time and text data enhancement effect. Summary of the invention

[0003] The embodiments of the present application provide a data enhancement method, device, electronic device and storage medium for text classification, which improve the enhancement effect of text data and achieve a good balance between event processing and text data enhancement effect.

[0004] In a first aspect, an embodiment of the present application provides a data enhancement method for text classification, comprising:

[0005] Obtain a training data set for text classification, wherein the training data set includes a true category label corresponding to each sample;

[0006] Determine the predicted category label corresponding to each sample in the training data set using a text classification model;

[0007] Based on the true category label and the predicted category label corresponding to each sample, a confusion matrix representing the classification accuracy of the text classification model is obtained;

[0008] Based on the confusion matrix, determining the enhancement probability corresponding to each category label;

[0009] For each category label in the training data set, according to the enhancement probability corresponding to the category label, determine the samples to be enhanced from the samples whose true category label is the category label, and perform corresponding data enhancement processing on each sample to be enhanced;

[0010] The samples corresponding to the training data set are replaced with the enhanced samples to obtain an enhanced training data set.

[0011] Optionally, the confusion matrix includes the number of samples under each true category label that are predicted to be samples of various category labels, and determining the enhancement probability corresponding to each category label based on the confusion matrix includes:

[0012] The enhanced probability corresponding to each category label is determined according to the following formula:

[0013] P i =1-S i ,

[0014] Among them, P i Represents the i-th category label L i The corresponding enhancement probability, e is a natural constant, Indicates that the true category label is L i The predicted class label in the sample is L i The proportion of samples Indicates that the true category label is L i The predicted class label in the sample is L j The percentage of samples in the training dataset is , and J is the number of category labels contained in the training dataset.

[0015] Optionally, the determining, according to the enhancement probability corresponding to the category label, the samples to be enhanced from the samples whose true category labels are the category labels includes:

[0016] For each sample whose true category label is the category label, determine whether the sample is a sample to be enhanced according to the enhancement probability corresponding to the category label; or,

[0017] Randomly select m samples from the samples whose true category labels are the category labels as samples to be enhanced, where m = P × n, P represents the enhancement probability corresponding to the category label, and n represents the total number of samples whose true category labels are the category labels in the training data set.

[0018] Optionally, performing corresponding data enhancement processing on each sample to be enhanced includes:

[0019] Determine the data enhancement method corresponding to each sample to be enhanced according to the true category label of each sample to be enhanced and the corresponding relationship between the category label and the data enhancement method;

[0020] According to the data enhancement method corresponding to each sample to be enhanced, data enhancement processing is performed on each sample to be enhanced.

[0021] Optionally, the determining the predicted category label corresponding to each sample in the training data set by using a text classification model includes:

[0022] Using the training data set to train the text classification model;

[0023] Each sample in the training data set is input into the trained text classification model to obtain the predicted category label corresponding to each sample.

[0024] Optionally, the training the text classification model using the training data set includes:

[0025] Performing multiple rounds of training on the text classification model using the training data set;

[0026] Based on the predicted category labels corresponding to the samples predicted by the text classification model obtained after each round of training, the text classification model obtained after each round of training is evaluated based on the predicted category labels and the true category labels;

[0027] The text classification model with the highest evaluation score is used as the trained text classification model.

[0028] In a second aspect, an embodiment of the present application provides a method for training a text classification model, comprising:

[0029] Based on any one of the methods described in the first aspect above, obtaining an enhanced training data set;

[0030] The text classification model is trained based on the enhanced training data set.

[0031] In a third aspect, an embodiment of the present application provides a data enhancement device for text classification, including:

[0032] An acquisition module is used to acquire a training data set for text classification, wherein the training data set includes a true category label corresponding to each sample;

[0033] A prediction module, used to determine a predicted category label corresponding to each sample in the training data set using a text classification model;

[0034] A confusion module, used for obtaining a confusion matrix representing the classification accuracy of the text classification model based on the true category label and the predicted category label corresponding to each sample;

[0035] An enhancement probability determination module, used to determine the enhancement probability corresponding to each category label based on the confusion matrix;

[0036] An enhancement module is used to determine, for each category label in the training data set, samples to be enhanced from samples whose true category label is the category label according to the enhancement probability corresponding to the category label, and perform corresponding data enhancement processing on each sample to be enhanced;

[0037] The replacement module is used to replace the samples corresponding to the training data set with the enhanced samples to obtain the enhanced training data set.

[0038] In a fourth aspect, an embodiment of the present application provides a training device for a text classification model, comprising:

[0039] An acquisition module is used to acquire a training data set for text classification, wherein the training data set includes a true category label corresponding to each sample;

[0040] A prediction module, used to determine a predicted category label corresponding to each sample in the training data set using a text classification model;

[0041] A confusion module, used for obtaining a confusion matrix representing the classification accuracy of the text classification model based on the true category label and the predicted category label corresponding to each sample;

[0042] An enhancement probability determination module, used to determine the enhancement probability corresponding to each category label based on the confusion matrix;

[0043] An enhancement module is used to determine, for each category label in the training data set, samples to be enhanced from samples whose true category label is the category label according to the enhancement probability corresponding to the category label, and perform corresponding data enhancement processing on each sample to be enhanced;

[0044] The replacement module is used to replace the samples corresponding to the training data set with the enhanced samples to obtain the enhanced training data set.

[0045] The training module is used to train the text classification model based on the enhanced training data set.

[0046] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the computer program.

[0047] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of any of the above methods when executed by a processor.

[0048] In a seventh aspect, an embodiment of the present application provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in any of the above-mentioned various optional implementations of controlling TCP transmission performance.

[0049] The solution provided in the embodiment of the present application uses a text classification model to determine the predicted category label corresponding to each sample in the training data set, and obtains a confusion matrix that characterizes the classification accuracy of the text classification model by comparing the real category label and the predicted category label of the sample. Based on the confusion matrix, the enhancement probability corresponding to each category label is obtained, and each category of samples is enhanced according to the enhancement probability, so that categories with a high confusion ratio that affect model performance receive more enhancement processing, thereby reasonably improving the enhancement effect of text data. Compared with existing data enhancement methods, relatively good results can be obtained without incurring a large computing cost, and a better balance is achieved in terms of computing time and data enhancement effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 A schematic diagram of an application scenario of a method for training a text classification model and data enhancement for text classification provided in an embodiment of the present application;

[0052] Figure 2 A flowchart of a data enhancement method for text classification provided in an embodiment of the present application;

[0053] Figure 3 A flowchart of a text classification model training method provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of the structure of a data enhancement device for text classification provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the structure of a training device for a text classification model provided in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0058] It should be noted that the following embodiments and features in the embodiments may be combined with each other in the absence of conflict; and, based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making any creative work are within the scope of protection of the present disclosure.

[0059] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0060] Data augmentation is a process of filling missing data into the original sample set, with the goal of reducing the gap between the augmented dataset and the real dataset. Taking multi-classification tasks as an example, excluding the quality issues of the samples themselves, the key point that affects the performance of artificial intelligence natural language models is that the samples are relatively similar but belong to different classification categories. These samples strongly interfere with each other and affect the judgment of the model.

[0061] To this end, the present application proposes a data enhancement method for text classification, which uses a text classification model to determine the predicted category label corresponding to each sample in the training data set, and obtains a confusion matrix that characterizes the classification accuracy of the text classification model by comparing the true category label and the predicted category label of the sample. Based on the confusion matrix, the enhancement probability corresponding to each category label is obtained, and each category of samples is enhanced according to the enhancement probability, so that categories with a high confusion ratio that affect model performance receive more enhancement processing, thereby reasonably improving the enhancement effect of text data. Compared with existing data enhancement methods, the solution of the present application can obtain relatively good results without incurring a large computing cost, and achieves a good balance between computing time and data enhancement effect.

[0062] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0063] refer to Figure 1, which is a schematic diagram of an application scenario of a data enhancement method for text classification provided in an embodiment of the present application. Among them, the terminal device 101 and the server 102 can be connected via a wireless or wired network, the terminal device 101 includes but is not limited to electronic devices such as desktop computers, mobile phones, mobile computers, tablet computers, media players, smart wearable devices, smart TVs, robots, etc., and the server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0064] The database of the server 102 can store a large amount of text data collected by the terminal device 101, such as data in the fields of news reports, papers, complaint tickets, etc. The specific field is determined by the type of service provided by the server 102. The category labeling of text data can be completed by manual labeling, and training data sets in various fields are obtained and stored in the database of the server 102. When it is necessary to train a text classification model in a certain field, the server 102 obtains the training data set of the field from the database, trains the text classification model based on the training data set of the field, and obtains a text classification model that can be applied to the field. Then the server 102 can provide the terminal device 101 with a text classification service in the field. For example, the server 102 can classify and store the text data in the field to improve the efficiency of data storage and retrieval, and can also display different categories on the terminal device 101 to facilitate users to retrieve corresponding data under different categories.

[0065] In the data retrieval scenario, the user can send the retrieval conditions to the server 102 through the terminal device 101. The server 102 quickly retrieves the data that meets the retrieval conditions from the database according to the retrieval conditions and the category tags, and feeds back to the terminal device 101. In the data push scenario, the server 102 can determine the user preference according to the user information, and determine at least one category tag that matches the user preference, select the data to be pushed to the user from the data under the category tag, and push the data to the user's terminal device 101.

[0066] Of course, the method provided in the embodiment of the present application is not limited to Figure 1 The application scenarios shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto. Figure 1 The functions that can be implemented by each device in the application scenario shown will be described in the subsequent method embodiments, and will not be described in detail here.

[0067] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present application.

[0068] Combine the following Figure 1 The application scenarios shown illustrate the technical solutions provided by the embodiments of the present application. Any data enhancement method for text classification and training method for text classification model provided by the embodiments of the present application can be applied to electronic devices with computing capabilities such as terminal devices or servers, and the present application does not limit this.

[0069] refer to Figure 2 The present invention provides a data enhancement method for text classification, which specifically includes the following steps:

[0070] S201. Obtain a training data set for text classification, where the training data set includes a true category label corresponding to each sample.

[0071] It should be noted that the samples in the training data set are texts that have been annotated with category labels. For the convenience of description, the annotated category labels are recorded as true category labels. The training data set can be a data set for text classification in a specific field.

[0072] S202: Determine the predicted category label corresponding to each sample in the training data set using the text classification model.

[0073] In specific implementation, the text classification model can adopt existing machine learning models, neural network models, etc. For example, the text classification model can be the tiny-BERT Chinese pre-trained model, which takes the vector of the [CLS] dimension of the last hidden layer and connects it to a 2-layer fully connected network. The output dimension of the first layer is twice the number of labels, the output dimension of the second layer is the number of predicted labels, and the activation function is softmax.

[0074] In specific implementation, existing vectorization methods (such as word2vec, BERT, xlnet, electra and other models) can be used to convert samples into encoding vectors, and the encoding vectors are input into the text classification model to obtain the corresponding predicted category labels.

[0075] In a possible implementation, the trained text classification model can be directly used to obtain the predicted category label corresponding to the sample. Specifically, the sample in the training data set is input into the text classification model, and the category label of the input sample is predicted by the text classification model, which is recorded as the predicted category label to distinguish it from the true category label.

[0076] In another possible implementation, the text classification model can be trained using the training data set to obtain a text classification model with better classification effect, thereby improving the accuracy of the model input prediction category label. To this end, step S202 specifically includes the following steps: training the text classification model using the training data set; inputting each sample in the training data set into the trained text classification model to obtain the predicted category label corresponding to each sample. Among them, the training method for the text classification model can adopt the existing model training method.

[0077] In specific implementation, in order to improve the accuracy of predicted category labels, the training data set can be used to perform multiple rounds of training on the text classification model; based on the predicted category labels corresponding to the samples predicted by the text classification model obtained after each round of training, the text classification model obtained after each round of training is evaluated based on the predicted category labels and the true category labels; the text classification model with the highest evaluation score is used as the trained text classification model.

[0078] In a possible example, the original training data set is cleaned and logically verified to form a relatively standard training data set D. Then, the training data set D is split into training sets D according to a ratio of 8:2. train and validation set D eval , we need to ensure that the training set D train and validation set D eval The number of sample labels in the training dataset D is roughly the same as that in the training dataset D. Then, we use the training dataset D train Train the text classification model. After training, the validation set D will be used. eval Verify the trained text classification model by comparing the prediction results of the text classification model input with the verification set D eval The real category label is used to obtain the F1 score of the text classification model obtained in this round of training (F1Score is an indicator used in statistics to measure the accuracy of binary classification or multi-task binary classification models). Then, the training data set D is split into a new training set D according to the ratio of 8:2. train and validation set D eval, conduct the next round of training for the text classification model and obtain the corresponding F1 score. After multiple rounds of training, take the text classification model with the highest F1 score in multiple rounds of training as the trained text classification model, and input each sample in the training data set into the trained text classification model to obtain the predicted category label corresponding to each sample.

[0079] S203: Based on the true category label and the predicted category label corresponding to each sample, a confusion matrix representing the classification accuracy of the text classification model is obtained.

[0080] The confusion matrix includes the number of samples under each true category label that are predicted to be various category labels.

[0081] For example, the training data set includes 4 categories, each category includes 100 samples, and the true category labels of the 4 categories are and Indicates that the predicted category labels of the four categories are respectively and Indicates that the predicted category label corresponding to each sample in the training data set is determined by using the text classification model, and the confusion matrix is ​​obtained as shown in Table 1. Taking the number 55 in the second row and second column of Table 1 as an example, it means that the label is The number of samples predicted to be class 1 is 55, and the number 55 in the second row and third column indicates that the label is The number of samples predicted as category 2 is 12. By analyzing the number of confused samples where the true category label is inconsistent with the predicted category label, we found that the categories with higher confusion ratios are This means that the confusion probability of samples in categories L1 and L4 is high, which will affect the model performance and these samples need to be enhanced.

[0082] Table 1

[0083]

[0084] S204: Based on the confusion matrix, determine the enhancement probability corresponding to each category label.

[0085] The enhancement probability refers to the probability that a sample needs to be enhanced. For example, if the enhancement probability of a certain category label is 30%, then there is a 30% probability that the samples under this category label will be enhanced. i The more samples are predicted to be other classes, the more likely class L is i The higher the probability of needing enhancement processing, the higher the enhancement probability.

[0086] In specific implementation, the enhancement probability corresponding to each category label can be determined according to the following formula:

[0087] P i =1-S i ,

[0088] Among them, P i Represents the i-th category label L i The corresponding enhancement probability, e is a natural constant, Indicates that the true category label is L i The predicted class label in the sample is L i The proportion of samples Indicates that the true category label is L i The predicted class label in the sample is L j The percentage of samples in the training dataset is , and J is the number of category labels contained in the training dataset.

[0089] Taking the data in Table 1 as an example, the enhanced probability of category label L1 is The enhanced probability of category label L2 is The enhanced probability of category label L3 is The enhanced probability of category label L4 is

[0090] S205. For each category label in the training data set, according to the enhancement probability corresponding to the category label, determine the samples to be enhanced from the samples whose true category label is the category label, and perform corresponding data enhancement processing on each sample to be enhanced.

[0091] In a possible implementation, for each sample whose true category label is the category label, whether the sample is a sample to be enhanced may be determined according to the enhancement probability corresponding to the category label.

[0092] For example, the enhancement probability of the category label L1 is 40%. For samples whose real category label is L1, a random algorithm such as roulette or dice rolling can be used to determine whether the sample needs to be enhanced. The random algorithm is set to: the probability that the sample is selected for enhancement is 40%. Therefore, there is a 40% probability that the sample whose real category label is L1 will be enhanced.

[0093] In another possible implementation, m samples are randomly selected from samples whose true category labels are the category labels as samples to be enhanced, where m = P × n, P represents the enhancement probability corresponding to the category label, and n represents the total number of samples whose true category labels are the category labels in the training data set.

[0094] For example, the enhancement probability of the category label L1 is 40%, and the total number of samples with the true category label L1 is 100, then m = 100 × 40% = 40, 40 samples are randomly selected from these 100 samples, and these samples are enhanced, and the remaining 60 samples are not enhanced.

[0095] In specific implementation, a data enhancement method that meets the sample characteristics can be configured for each category. For example, the sample with label L1 is enhanced by method A1, and the sample with label L4 is enhanced by method A4. The data enhancement methods include but are not limited to:

[0096] (1) First translate the sample (assuming the sample is Chinese) into English, then translate English into French, and finally translate French into Chinese.

[0097] (2) Add random noise to the sample. For example, each byte is independent and identically distributed, with a probability of 10% replacing adjacent words and a probability of 15% replacing synonyms.

[0098] (3) Replacement of synonyms and antonyms.

[0099] (4) Replacement of content of the same entity.

[0100] Specifically, the data enhancement method corresponding to each sample to be enhanced can be determined based on the true category label of each sample to be enhanced, and the correspondence between the category label and the data enhancement method; then, data enhancement processing is performed on each sample to be enhanced according to the data enhancement method corresponding to each sample to be enhanced. For example, the sample with label L1 is enhanced by enhancement method A1, and the sample with label L4 is enhanced by enhancement method A4. Then, the sample to be enhanced whose true category label is label L1 is enhanced by enhancement method A1, and the sample to be enhanced whose true category label is label L4 is enhanced by enhancement method A4.

[0101] S206: Replace the samples corresponding to the training data set with the enhanced samples to obtain an enhanced training data set.

[0102] It should be noted that for the samples that are enhanced in the training data set, the original samples in the training data set need to be replaced with the enhanced samples; for the samples that are not enhanced in the training data set, no replacement is required and the samples can be retained.

[0103] Of course, after obtaining the enhanced training data set, the above steps S202 to S206 may be repeated on the enhanced training data set to obtain a better quality training data set.

[0104] The method of the embodiment of the present application analyzes the number of confused samples between the real category label and the predicted category label, determines the enhancement probability of each category according to the number of confused samples of each category, and enhances the samples of each category according to the enhancement probability, so that categories with high confusion ratios that affect model performance receive more enhancement processing, which can reasonably improve the enhancement effect of text data.

[0105] Assume that the existing automatic optimization strategy Auto-P obtains the optimal result Res after O(n) times, usually O(n) is a multiple of its search space. For example, the automatic optimization space is three data enhancement methods A1, A2, and A3, and the corresponding enhancement parameter values ​​are selected as discrete values, with the number of 2, 3, and 4 respectively. Then the automatic optimization calculation complexity is 2×3×4. Considering the single calculation time T, the total time cost is 24T. The scheme of the embodiment of the present application is not sensitive to the various parameter values ​​of the enhancement method of the automatic optimization strategy. Usually, a default value can be specified, and its complexity is 2T, that is, only 2 training times are required to obtain the optimal parameters. Therefore, compared with the automatic optimization strategy, the method of the embodiment of the present application can greatly reduce the calculation time while also achieving results that are higher than the global enhancement effect, and it strikes a good balance between the calculation time and the text data enhancement effect.

[0106] Based on the data enhancement method for text classification provided in the above embodiment, the present application embodiment also provides a training method for a text classification model, referring to Figure 3 , specifically including the following steps:

[0107] S301. Obtain a training data set for text classification, where the training data set includes a true category label corresponding to each sample.

[0108] S302: Determine the predicted category label corresponding to each sample in the training data set using the text classification model.

[0109] S303: Based on the true category label and the predicted category label corresponding to each sample, a confusion matrix representing the classification accuracy of the text classification model is obtained.

[0110] S304: Determine the enhancement probability corresponding to each category label based on the confusion matrix.

[0111] S305 . For each category label in the training data set, according to the enhancement probability corresponding to the category label, determine the samples to be enhanced from the samples whose true category label is the category label, and perform corresponding data enhancement processing on each sample to be enhanced.

[0112] S306: Replace the samples corresponding to the training data set with the enhanced samples to obtain an enhanced training data set.

[0113] S307: Training a text classification model based on the enhanced training data set.

[0114] Among them, the specific implementation of S301 to S306 can refer to the specific implementation of the data enhancement method for text classification, and will not be repeated here.

[0115] By training the text classification model with the enhanced training data set, the impact of easily confused samples on the model performance can be reduced and the classification accuracy of the text classification model can be improved.

[0116] The following is an example of a multi-classification task in a banking business in the financial field to illustrate the method provided in the embodiment of the present application:

[0117] The categories of appeal tickets include: Business A complaint, Business B complaint, Business C complaint, Business A consultation, Business B consultation, Business C consultation, etc.

[0118] First, after obtaining a large number of claim work orders, data cleaning and data logic verification are performed on the claim work orders, and the categories of the claim work orders are labeled to obtain a standard training data set D. Each sample in the training data set includes the description text of the claim work order and the labeled category label (i.e., the true category label).

[0119] Then, the training data set D is split into two data sets D according to the ratio of 8:2 train and validation set D eval Use D train Train the tiny-BERT Chinese pre-trained model. The number of training rounds is 8. After each round of training, the model prediction D eval The predicted category labels of the samples in the training dataset D are compared with the true category labels to calculate the F1 score. The model with the highest F1 score in the 8 rounds is used to obtain the predicted category labels of each sample in the training dataset D based on the model, and the confusion matrix is ​​made with the true sample labels, and the enhanced probability corresponding to each category label is calculated.

[0120] For each category label in the training data set D, according to the enhancement probability corresponding to the category label, the sample to be enhanced is determined from the samples whose true category label is the category label, and the corresponding data enhancement processing is performed on each sample to be enhanced. Among them, an enhancement method is applied to each confused label, for example, the sample text of label L1 is applied with enhancement method A1, and the sample text of label L4 is applied with enhancement method A4. The original text in the training data set D is replaced with the enhanced text to obtain the enhanced training data set.

[0121] The model with the highest F1 score is trained with the enhanced training dataset to obtain the final target model.

[0122] like Figure 4 As shown, based on the same inventive concept as the above-mentioned data enhancement method for text classification, the embodiment of the present application further provides a data enhancement device 40 for text classification, including:

[0123] An acquisition module 401 is used to acquire a training data set for text classification, where the training data set includes a true category label corresponding to each sample;

[0124] Prediction module 402, used to determine the predicted category label corresponding to each sample in the training data set using a text classification model;

[0125] A confusion module 403 is used to obtain a confusion matrix representing the classification accuracy of the text classification model based on the true category label and the predicted category label corresponding to each sample;

[0126] An enhancement probability determination module 404 is used to determine the enhancement probability corresponding to each category label based on the confusion matrix;

[0127] The enhancement module 405 is used to determine, for each category label in the training data set, samples to be enhanced from samples whose true category label is the category label according to the enhancement probability corresponding to the category label, and perform corresponding data enhancement processing on each sample to be enhanced;

[0128] The replacement module 406 is configured to replace the samples corresponding to the training data set with the enhanced samples to obtain an enhanced training data set.

[0129] Optionally, the confusion matrix includes the number of samples under each true category label that are predicted to be samples of various category labels, and the enhancement probability determination module 404 is specifically used to determine the enhancement probability corresponding to each category label according to the following formula:

[0130] P i =1-S i ,

[0131] Among them, P i Represents the i-th category label L i The corresponding enhancement probability, e is a natural constant, Indicates that the true category label is L i The predicted class label in the sample is L i The proportion of samples Indicates that the true category label is L i The predicted class label in the sample is L j The percentage of samples in the training dataset is , and J is the number of category labels contained in the training dataset.

[0132] Optionally, the enhancement module 405 is specifically configured to: for each sample whose true category label is the category label, determine whether the sample is a sample to be enhanced according to the enhancement probability corresponding to the category label.

[0133] Optionally, the enhancement module 405 is specifically used to randomly select m samples from samples whose true category labels are the category labels as samples to be enhanced, where m = P × n, P represents the enhancement probability corresponding to the category label, and n represents the total number of samples whose true category labels are the category labels in the training data set.

[0134] Optionally, the enhancement module 405 is specifically used to: determine the data enhancement method corresponding to each sample to be enhanced according to the true category label of each sample to be enhanced, and the correspondence between the category label and the data enhancement method; and perform data enhancement processing on each sample to be enhanced according to the data enhancement method corresponding to each sample to be enhanced.

[0135] Optionally, the prediction module 402 is specifically used to: train the text classification model using the training data set; input each sample in the training data set into the trained text classification model to obtain a predicted category label corresponding to each sample.

[0136] Optionally, the prediction module 402 is specifically used to: perform multiple rounds of training on the text classification model using the training data set; predict the predicted category labels corresponding to the samples obtained after each round of training based on the text classification model, and evaluate the text classification model obtained after each round of training based on the predicted category labels and the true category labels; and use the text classification model with the highest evaluation score as the trained text classification model.

[0137] The data enhancement device for text classification proposed in the embodiment of the present application adopts the same inventive concept as the above-mentioned data enhancement method for text classification and can achieve the same beneficial effects, which will not be repeated here.

[0138] like Figure 5 As shown, based on the data enhancement device 40 for text classification, the embodiment of the present application further provides a text classification model training device 50, including:

[0139] An acquisition module 401 is used to acquire a training data set for text classification, where the training data set includes a true category label corresponding to each sample;

[0140] Prediction module 402, used to determine the predicted category label corresponding to each sample in the training data set using a text classification model;

[0141] A confusion module 403 is used to obtain a confusion matrix representing the classification accuracy of the text classification model based on the true category label and the predicted category label corresponding to each sample;

[0142] An enhancement probability determination module 404 is used to determine the enhancement probability corresponding to each category label based on the confusion matrix;

[0143] The enhancement module 405 is used to determine, for each category label in the training data set, samples to be enhanced from samples whose true category label is the category label according to the enhancement probability corresponding to the category label, and perform corresponding data enhancement processing on each sample to be enhanced;

[0144] The replacement module 406 is configured to replace the samples corresponding to the training data set with the enhanced samples to obtain an enhanced training data set.

[0145] The training module 407 is used to train the text classification model based on the enhanced training data set.

[0146] The training device for the text classification model proposed in the embodiment of the present application and the training method for the above-mentioned text classification model adopt the same inventive concept and can achieve the same beneficial effects, which will not be repeated here.

[0147] Based on the same inventive concept as the above-mentioned data enhancement method for text classification and the training method of the text classification model, the embodiment of the present application also provides an electronic device, which can be a control device or control system inside a robot, or an external device that communicates with an intelligent device, such as a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), a server, etc. Figure 6 As shown, the electronic device 60 may include a processor 601 and a memory 602 .

[0148] Processor 601 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware processor to be executed, or can be executed by a combination of hardware and software modules in the processor.

[0149] The memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0150] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; the above computer storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to: mobile storage devices, random access memory (RAM, Random Access Memory), magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)) and other media that can store program codes.

[0151] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a mobile storage device, a random access memory (RAM, Random Access Memory), a magnetic storage device (such as a floppy disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical storage device (such as CD, DVD, BD, HVD, etc.), and a semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), a solid-state drive (SSD)) and other various media that can store program codes.

[0152] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A data enhancement method for text classification, characterized in that: include: Obtain a training data set for text classification, wherein the training data set includes a true category label corresponding to each sample; Determine the predicted category label corresponding to each sample in the training data set using a text classification model; Based on the true category label and the predicted category label corresponding to each sample, a confusion matrix representing the classification accuracy of the text classification model is obtained; Based on the confusion matrix, determining the enhancement probability corresponding to each category label; For each category label in the training data set, according to the enhancement probability corresponding to the category label, determine the samples to be enhanced from the samples whose true category label is the category label, and perform corresponding data enhancement processing on each sample to be enhanced; Replace the samples corresponding to the training data set with the enhanced samples to obtain an enhanced training data set; The confusion matrix includes the number of samples under each true category label that are predicted to be samples of various category labels, and determining the enhancement probability corresponding to each category label based on the confusion matrix includes: The enhanced probability corresponding to each category label is determined according to the following formula: ， ， in, Indicates i Category labels L i The corresponding enhancement probability is e is a natural constant, Indicates that the true category label is L i The predicted class label in the sample is L i The proportion of samples Indicates that the true category label is L i The predicted class label in the sample is L j The percentage of samples in the training dataset is , and J is the number of category labels contained in the training dataset.

2. The method according to claim 1, characterized in that The step of determining the samples to be enhanced from the samples whose real category labels are the category labels according to the enhancement probability corresponding to the category label includes: For each sample whose true category label is the category label, determine whether the sample is a sample to be enhanced according to the enhancement probability corresponding to the category label; or, Randomly select m samples from the samples whose true category labels are the category labels as samples to be enhanced, where m=P×n, P represents the enhancement probability corresponding to the category label, and n represents the total number of samples whose true category labels are the category labels in the training data set.

3. The method according to claim 1, characterized in that The corresponding data enhancement processing is performed on each sample to be enhanced, including: Determine the data enhancement method corresponding to each sample to be enhanced according to the true category label of each sample to be enhanced and the corresponding relationship between the category label and the data enhancement method; According to the data enhancement method corresponding to each sample to be enhanced, data enhancement processing is performed on each sample to be enhanced.

4. The method according to any one of claims 1 to 3, characterized in that: The method of using the text classification model to determine the predicted category label corresponding to each sample in the training data set includes: Using the training data set to train the text classification model; Each sample in the training data set is input into the trained text classification model to obtain the predicted category label corresponding to each sample.

5. A training method for a text classification model, characterized in that: Based on the method described in any one of claims 1 to 4, obtaining an enhanced training data set; The text classification model is trained based on the enhanced training data set.

6. A data enhancement device for text classification, characterized in that: include: An acquisition module is used to acquire a training data set for text classification, wherein the training data set includes a true category label corresponding to each sample; A prediction module, used to determine a predicted category label corresponding to each sample in the training data set using a text classification model; A confusion module, used to obtain a confusion matrix representing the classification accuracy of the text classification model based on the true category label and the predicted category label corresponding to each sample; An enhancement probability determination module, used to determine the enhancement probability corresponding to each category label based on the confusion matrix; An enhancement module is used to determine, for each category label in the training data set, samples to be enhanced from samples whose true category label is the category label according to the enhancement probability corresponding to the category label, and perform corresponding data enhancement processing on each sample to be enhanced; A replacement module, used to replace samples corresponding to the training data set with enhanced samples to obtain an enhanced training data set; The confusion matrix includes the number of samples under each true category label that are predicted to be samples of various category labels, and determining the enhancement probability corresponding to each category label based on the confusion matrix includes: The enhanced probability corresponding to each category label is determined according to the following formula: ， ， in, Indicates i Category labels L i The corresponding enhancement probability is e is a natural constant, Indicates that the true category label is L i The predicted class label in the sample is L i The proportion of samples Indicates that the true category label is L i The predicted class label in the sample is L j The percentage of samples in the training dataset is , and J is the number of category labels contained in the training dataset.

7. A training device for a text classification model, characterized in that: include: An acquisition module is used to acquire a training data set for text classification, wherein the training data set includes a true category label corresponding to each sample; A prediction module, used to determine a predicted category label corresponding to each sample in the training data set using a text classification model; A confusion module, used to obtain a confusion matrix representing the classification accuracy of the text classification model based on the true category label and the predicted category label corresponding to each sample; An enhancement probability determination module, used to determine the enhancement probability corresponding to each category label based on the confusion matrix; An enhancement module is used to determine, for each category label in the training data set, samples to be enhanced from samples whose true category label is the category label according to the enhancement probability corresponding to the category label, and perform corresponding data enhancement processing on each sample to be enhanced; A replacement module, used to replace samples corresponding to the training data set with enhanced samples to obtain an enhanced training data set; A training module, used for training the text classification model based on the enhanced training data set; The confusion matrix includes the number of samples under each true category label that are predicted to be samples of various category labels, and determining the enhancement probability corresponding to each category label based on the confusion matrix includes: The enhanced probability corresponding to each category label is determined according to the following formula: ， ， in, Indicates i Category labels L i The corresponding enhancement probability is e is a natural constant, Indicates that the true category label is L i The predicted class label in the sample is L i The proportion of samples Indicates that the true category label is L i The predicted class label in the sample is L j The percentage of samples in the training dataset is , and J is the number of category labels contained in the training dataset.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images