Methods and apparatuses for text classification and classification model training based on label smoothing

Abstract

The present invention discloses a method and apparatus for text classification and text classification model training based on label smoothing. The method determines a hierarchical industry object to which the text of the text category to be recognized belongs; and inputs all the categories existing in the lowest level of the text and the industry object into a text classification model trained by sample texts with smoothed corresponding label values and a preset loss layer for analysis, and determines the category corresponding to the index to which the maximum label value analyzed corresponds as the category of the text, which can improve the analysis accuracy of the specific category to which the text belongs, obtain accurate multi-level labels, so that the classified labels can accurately express the text meaning and facilitate the classification and induction of the text, and further contribute to improving the availability of the text.

Description

Technical Field

[0001] The present invention relates to the technical field of text classification, and in particular, to a method and device for text classification and classification model training based on label smoothing. Background Art

[0002] Text classification is of great significance for identifying the precise meaning expressed by texts. Currently, it is usually to perform single-level label classification on texts, that is, only assign a single-level label to a piece of text. For example, the text "xx men's leather shoes" is classified as clothing, shoes and bags.

[0003] However, it is found in practice that the accuracy of the text labels identified by the existing single-level label classification method is very low, resulting in the classified labels being unable to accurately express the text meaning, which is not conducive to classifying and summarizing texts. Therefore, it is particularly important to propose a technical solution on how to improve the accuracy of text classification, so that the classified labels can accurately express the text meaning and facilitate classifying and summarizing texts. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and device for text classification and text classification model training based on label smoothing, which can improve the accuracy of text classification, so that the classified labels can accurately express the text meaning and facilitate classifying and summarizing texts.

[0005] To solve the above technical problem, the first aspect of the present invention discloses a text classification method based on label smoothing, and the method includes:

[0006] Determine the target industry object to which the target text of the text category to be recognized belongs. There are multiple target levels for the target industry object, and there are multiple categories in the lowest-level target level among all the target levels, and the number of each category is greater than or equal to 1;

[0007] Input the target text and all the categories existing in the lowest-level target level of the target industry object into a pre-trained text classification model and a preset loss layer for analysis. The text classification model is a model trained by sample texts with smoothed corresponding label values;

[0008] Obtain the analysis result output by the text classification model and the preset loss layer. The analysis result includes the label value corresponding to the index of each category of the target text in the lowest-level target level, and determine the category corresponding to the index of the maximum label value as the category of the target text according to the label value corresponding to each category.

[0009] As an optional implementation manner, in the first aspect of the present invention, the method further includes:

[0010] Determine a set of sample texts corresponding to the sample industry object. The sample industry object has multiple levels. There are multiple categories in the lowest level among all the levels, and the number of each category is greater than or equal to 1. The set of sample texts contains multiple sample texts;

[0011] Determine the target category to which each sample text belongs in the lowest level, and determine the total loss of each sample text according to the target category to which each sample text belongs, the number of all levels, each category, and the determined label smoothing coefficient;

[0012] Train a basic text classification model based on the total losses of all the sample texts until the basic text classification model converges, obtain the converged basic text classification model, and determine the converged basic text classification model as the pre-trained text classification model.

[0013] As an optional implementation manner, in the first aspect of the present invention, determining the total loss of each sample text according to the target category to which each sample text belongs, the number of all levels, each category, and the determined label smoothing coefficient includes:

[0014] For any one of the sample texts:

[0015] Determine the correlation between each category and the target category to which the sample text belongs;

[0016] Based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all levels, determine the smoothed label value of each category;

[0017] Input the smoothed label value of each category into the basic text classification model and the loss layer for analysis to obtain the propensity score of each category corresponding to the sample text;

[0018] Determine the total loss of the sample text based on the propensity score of each category corresponding to the sample text and the smoothed label value of this category;

[0019] The calculation formula for the total loss of each sample text is as follows:

[0020]

[0021] In the formula, the loss represents the total loss of each sample text, p j represents the smoothed label value of the jth category, logq jDenote the propensity score corresponding to the smoothed label value of the j-th category, and k represents the total number of all categories in the lowest level;

[0022] The calculation method of the smoothed label value of each category in each of the said levels can be shown as follows:

[0023]

[0024] In the formula, the said target represents the smoothed label value of each category at the z-th level, and the is the label smoothing coefficient, the N is the number of all the said levels of the sample industry object, and the N z represents the label value allocation ratio, the P z represents the number of corresponding categories at the z-th level, the Hz(j) represents obtaining the index of the j-th category at the z-th level, and the Hz(i) represents obtaining the index of the target category i at the z-th level.

[0025] As an optional implementation manner, in the first aspect of the present invention, determining the smoothed label value of each category corresponding to the sample text based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all the said levels includes:

[0026] Based on the number of all the said levels and the correlation corresponding to each category, determine the label value allocation ratio of all categories, and based on the determined label smoothing coefficient, determine the initial label value of the index where the target category to which the sample text belongs is located;

[0027] According to the correlation corresponding to each category, determine, from all categories, all categories that belong to the same level as the target category to which the sample text belongs, where the number of the said level is greater than or equal to 1 and less than or equal to the number of all the said levels, and the value of the said level is a positive integer;

[0028] Based on the label smoothing coefficient, the label value allocation ratio of all categories, and the number of each category corresponding to each of the said levels, determine the smoothed label value of each category corresponding to each of the said levels, and determine the initial label value corresponding to the sample text and the smoothed label value of each category corresponding to each of the said levels in all the said levels as the smoothed label value of each category corresponding to the sample text.

[0029] As an optional implementation manner, in the first aspect of the present invention, the method further includes:

[0030] Determine the index position of the target category to which the sample text belongs, and determine the one-hot code label value of the sample text according to the index position of the target category to which the sample text belongs, the index position of each category in all categories other than the target category, and the determined one-hot code strategy;

[0031] Among them, the determining the smoothed label value of each category corresponding to each certain level based on the label smoothing coefficient, the label value distribution ratio of all categories, and the number of all categories corresponding to each certain level includes:

[0032] Based on the one-hot code label value of the sample text, the label smoothing coefficient, the label value distribution ratio of all categories, and the number of all categories corresponding to each certain level, determine the smoothed label value of each category corresponding to each certain level.

[0033] As an optional implementation manner, in the first aspect of the present invention, after determining the correlation between each category and the target category to which the sample text belongs, the method further includes:

[0034] Determine the industry type of the sample industry object, and according to the industry type of the sample industry object and the correlation corresponding to each category, screen all categories whose correlation is greater than or equal to a preset correlation from all categories, and trigger the execution of the operation of determining the smoothed label value of each category corresponding to the sample text based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all levels;

[0035] Among them, each such category is a category whose correlation is greater than or equal to the preset correlation, and the preset correlation is determined by the industry type of the sample industry object.

[0036] As an optional implementation manner, in the first aspect of the present invention, the determining the correlation between each category and the target category to which the sample text belongs includes:

[0037] Analyze the hierarchical relationship between each category and the target category to which the sample text belongs, and determine the correlation between each category and the target category to which the sample text belongs according to the hierarchical relationship corresponding to each category.

[0038] The second aspect of the present invention discloses a classification model training method based on label smoothing, and the method includes:

[0039] Determine a set of sample texts corresponding to a sample industry object. The sample industry object has multiple levels. Among all the levels, the lowest-level one has multiple categories, and the number of each category is greater than or equal to 1. The set of sample texts contains multiple sample texts;

[0040] Determine the target category to which each sample text belongs in the lowest-level hierarchy, and determine the total loss of each sample text based on the target category to which each sample text belongs, the number of all the hierarchies, each category, and the determined label smoothing coefficient;

[0041] Based on the total loss of all the sample texts, train a basic text classification model until the basic text classification model converges, obtain the converged basic text classification model, and determine the converged basic text classification model as the pre-trained text classification model.

[0042] The third aspect of the present invention discloses a text classification device based on label smoothing. The device includes:

[0043] A determination module, configured to determine a target industry object to which a target text of a to-be-recognized text category belongs. The target industry object has multiple target levels. Among all the target levels, the lowest-level one has multiple categories, and the number of each category is greater than or equal to 1;

[0044] An analysis module, configured to input the target text and all the categories existing in the lowest-level target level of the target industry object into a pre-trained text classification model and a preset loss layer for analysis. The text classification model is a model trained with sample texts whose corresponding label values are smoothed;

[0045] An acquisition module, configured to acquire an analysis result output by the text classification model and the preset loss layer. The analysis result includes label values corresponding to indices of each category to which the target text belongs in the lowest-level target level;

[0046] The determination module is further configured to determine, according to the label value corresponding to each category, the category corresponding to the index corresponding to the maximum label value as the category of the target text.

[0047] As an optional implementation manner, in the third aspect of the present invention, the determination module is further configured to determine a set of sample texts corresponding to a sample industry object. The sample industry object has multiple levels. Among all the levels, the lowest-level one has multiple categories, and the number of each category is greater than or equal to 1. The set of sample texts contains multiple sample texts;

[0048] The determining module is further configured to determine the target category to which each of the sample texts belongs in the lowest-level hierarchy;

[0049] The determining module is further configured to determine the total loss of each of the sample texts according to the target category to which each of the sample texts belongs, the number of all the hierarchies, each category, and the determined label smoothing coefficient;

[0050] The apparatus further includes:

[0051] A training module, configured to train a basic text classification model based on the total loss of all the sample texts until the basic text classification model converges, so as to obtain the converged basic text classification model;

[0052] The determining module is further configured to determine the converged basic text classification model as the pre-trained text classification model.

[0053] As an optional implementation manner, in the third aspect of the present invention, the specific manner in which the determining module determines the total loss of each of the sample texts according to the target category to which each of the sample texts belongs, the number of all the hierarchies, each category, and the determined label smoothing coefficient includes:

[0054] For any one of the sample texts:

[0055] Determine the correlation between each category and the target category to which the sample text belongs;

[0056] Based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all the hierarchies, determine the smoothed label value of each category;

[0057] Input the smoothed label value of each category into the basic text classification model and the loss layer for analysis, so as to obtain the propensity score of each category corresponding to the sample text;

[0058] Based on the propensity score of each category corresponding to the sample text and the smoothed label value of this category, determine the total loss of the sample text;

[0059] The calculation formula for the total loss of each of the sample texts is as follows:

[0060]

[0061] In the formula, the loss represents the total loss of each of the sample texts, and p j represents the smoothed label value of the jth category, and logq jIt represents the propensity score corresponding to the smoothed label value of the j-th category, and k represents the total number of all categories in the lowest level;

[0062] The calculation method of the smoothed label value of each category in each level can be shown as follows:

[0063]

[0064] In the formula, the target represents the smoothed label value of each category at the z-th level, and the is the label smoothing coefficient, N is the number of all levels of the sample industry object, and the N z represents the label value allocation ratio, and the P z represents the number of corresponding categories at the z-th level, Hz(j) represents obtaining the index of the j-th category at the z-th level, and Hz(i) represents obtaining the index of the target category i at the z-th level.

[0065] As an optional implementation manner, in the third aspect of the present invention, the specific manner in which the determination module determines the smoothed label value of each category corresponding to the sample text based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all levels includes:

[0066] Based on the number of all levels and the correlation corresponding to each category, determine the label value allocation ratio of all categories, and based on the determined label smoothing coefficient, determine the initial label value of the index where the target category to which the sample text belongs is located;

[0067] According to the correlation corresponding to each category, determine all categories that belong to the same level as the target category to which the sample text belongs from all categories, where the number of the certain level is greater than or equal to 1 and less than or equal to the number of all levels, and the value of the certain level is a positive integer;

[0068] Based on the label smoothing coefficient, the label value allocation ratio of all categories, and the number of all categories corresponding to each certain level, determine the smoothed label value of each category corresponding to each certain level, and determine the initial label value corresponding to the sample text and the smoothed label value of each category corresponding to each certain level in all certain levels as the smoothed label value of each category corresponding to the sample text.

[0069] As an alternative implementation, in the third aspect of the present invention, the determining module is further configured to determine the index position of the target category to which the sample text belongs, and determine the one-hot code label value of the sample text according to the index position of the target category to which the sample text belongs, the index positions of each category in all categories other than the target category, and the determined one-hot code strategy;

[0070] Wherein, the specific manner in which the determining module determines the smoothed label value of each category corresponding to each certain level based on the label smoothing coefficient, the label value distribution ratio of all categories, and the number of all categories corresponding to each certain level includes:

[0071] Based on the one-hot code label value of the sample text, the label smoothing coefficient, the label value distribution ratio of all categories, and the number of all categories corresponding to each certain level, determine the smoothed label value of each category corresponding to each certain level.

[0072] As an alternative implementation, in the third aspect of the present invention, the determining module is further configured to determine the industry type of the sample industry object after determining the correlation between each category and the target category to which the sample text belongs;

[0073] The device further includes:

[0074] A screening module, configured to screen, from all categories, all categories whose correlation is greater than or equal to a preset correlation according to the industry type of the sample industry object and the correlation corresponding to each category, and trigger the determining module to perform the operation of determining the smoothed label value of each category corresponding to the sample text based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all levels;

[0075] Wherein, each such category is a category whose correlation is greater than or equal to the preset correlation, and the preset correlation is determined by the industry type of the sample industry object.

[0076] As an alternative implementation, in the third aspect of the present invention, the specific manner in which the determining module determines the correlation between each category and the target category to which the sample text belongs includes:

[0077] Analyze the hierarchical relationship between each category and the target category to which the sample text belongs, and determine the correlation between each category and the target category to which the sample text belongs according to the hierarchical relationship corresponding to each category.

[0078] The fourth aspect of the present invention discloses a classification model training device based on label smoothing. The device includes:

[0079] A determination module, configured to determine a set of sample texts corresponding to a sample industry object. The sample industry object has multiple levels, there are multiple categories in the lowest level among all the levels, and the number of each category is greater than or equal to 1. The set of sample texts includes multiple sample texts;

[0080] The determination module is further configured to determine the target category to which each sample text belongs in the lowest level;

[0081] The determination module is further configured to determine the total loss of each sample text according to the target category to which each sample text belongs, the number of all levels, each category, and the determined label smoothing coefficient;

[0082] A training module, configured to train a basic text classification model based on the total loss of all the sample texts until the basic text classification model converges, and obtain the converged basic text classification model;

[0083] The determination module is further configured to determine the converged basic text classification model as the pre-trained text classification model.

[0084] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0085] In an embodiment of the present invention, a target industry object to which a target text for determining a category of a text to be recognized belongs is determined. There are multiple target levels for the target industry object. Among all the target levels, there are multiple categories at the lowest-level target level, and the number of each category is greater than or equal to 1. The target text and all the categories corresponding to the lowest-level target level in the target industry object are input into a pre-trained text classification model and a preset loss layer for analysis. The text classification model is a model trained with sample texts whose corresponding label values are smoothed. An analysis result output by the text classification model and the preset loss layer is obtained. The analysis result includes label values corresponding to indexes of each category of the target text at the lowest-level target level. According to the label values corresponding to each category, the category corresponding to the index corresponding to the maximum label value is determined as the category of the target text. It can be seen that the present invention inputs the text to be classified and all the categories at the lowest level in the industry object to which the text belongs into a text classification model trained with sample texts with label smoothing for analysis, and determines the category corresponding to the index corresponding to the maximum label value analyzed as the category of the text, which can improve the analysis accuracy of the specific category to which the text belongs, obtain accurate multi-level labels, so that the classified labels can accurately express the text meaning and facilitate the classification and induction of the text, and further contribute to improving the utilization rate of the text. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0087] Figure 1 is a flowchart of a text classification method based on label smoothing disclosed in an embodiment of the present invention;

[0088] Figure 2 is a flowchart of a classification model training method based on label smoothing disclosed in an embodiment of the present invention;

[0089] Figure 3 is a structural diagram of a text classification device based on label smoothing disclosed in an embodiment of the present invention;

[0090] Figure 4 is a structural diagram of another text classification device based on label smoothing disclosed in an embodiment of the present invention;

[0091] Figure 5 is a structural diagram of a classification model training device based on label smoothing disclosed in an embodiment of the present invention;

[0092] Figure 6 It is a schematic structural diagram of another text processing device based on label smoothing disclosed in an embodiment of the present invention;

[0093] Figure 7 It is an example schematic diagram of another text classification method based on label smoothing disclosed in an embodiment of the present invention. Detailed implementation manners

[0094] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0095] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or terminal including a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or terminals.

[0096] Referring to "embodiment" herein means that a specific feature, structure or characteristic described in connection with the embodiment may be included in at least one embodiment of the present invention. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0097] The present invention discloses a method and device for text classification and text classification model training based on label smoothing, which can analyze by inputting the text to be classified and all categories in the lowest level of the industry object to which the text belongs into a text classification model trained with a label-smoothed sample text, and determine the category corresponding to the index corresponding to the maximum label value analyzed as the category of the text, which can improve the analysis accuracy of the specific category to which the text belongs, obtain accurate multi-level labels, so that the classified labels can accurately express the text meaning and facilitate the classification and induction of the text, and thus is conducive to improving the utilization rate of the text. The following will be described in detail respectively.

[0098] Embodiment 1

[0099] Please refer toFigure 1 , Figure 1 is a schematic flowchart of a text classification method based on label smoothing disclosed in an embodiment of the present invention. Among them, Figure 1 the described method can be applied to a text classification device based on label smoothing, such as a server, a cloud platform, etc., which is not limited in the embodiments of the present invention. As Figure 1 shown, the text classification method based on label smoothing may include the following operations:

[0100] 101. Determine the target industry object to which the target text of the text category to be recognized belongs. The target industry object has multiple target levels. There are multiple categories in the lowest-level target level among all target levels, and the number of each category is greater than or equal to 1.

[0101] In the embodiments of the present invention, optionally, the target text is the text that needs to be analyzed for categories in any industry object. Further, the language type of the target text includes Chinese type and / or non-Chinese type (such as English type). Among them, the target industry object / industry object includes, but is not limited to, one of the clothing, shoes and bags industry, the catering industry, the fitness equipment industry, the agricultural product industry, the cosmetics industry, the skin care products industry, the game industry, and the mother and baby industry. Each industry object has multiple levels. For example, in the clothing, shoes and bags industry, clothing, shoes and bags - shoes - women's shoes - women's sports shoes - girls' sports shoes, there are 5 levels. Level 1 is girls' sports shoes, level 2 is women's sports shoes, level 3 is women's shoes, level 4 is shoes, and level 5 is clothing, shoes and bags. It should be noted that the larger the level number does not mean that the category range corresponding to the level is larger. On the contrary, level 1 is clothing, shoes and bags... level 5 is girls' sports shoes, which can all explain the present invention. At this time, the lowest-level is level 5. That is, the level is defined by the corresponding category range, that is, the smaller the category range, the lower the level it represents. For example, the range of women's sports shoes is larger than the range of girls' sports shoes.

[0102] 102. Input the target text and all categories corresponding to the lowest-level target level in the target industry object into a pre-trained text classification model and a preset loss layer for analysis. The text classification model is a model trained with sample texts whose corresponding label values are smoothed.

[0103] 103. Obtain the analysis result output by the text classification model and the preset loss layer. The analysis result includes the label values corresponding to the indexes of each category to which the target text belongs in the lowest-level target level.

[0104] 104. According to the label value corresponding to each category, determine the category corresponding to the index corresponding to the maximum label value as the category of the target text.

[0105] It can be seen that implementing Figure 1The described method can analyze by inputting the text to be classified and all categories in the lowest level of the industry object to which the text belongs into a text classification model trained with label-smoothed sample texts, and determine the category corresponding to the index corresponding to the maximum label value analyzed as the category of the text, which can improve the analysis accuracy of the specific category to which the text belongs, obtain accurate multi-level labels, so that the classified labels can accurately express the text meaning and facilitate the classification and induction of the text, and further contribute to improving the availability of the text.

[0106] In an optional embodiment, the method may further include the following steps:

[0107] When the number of categories corresponding to the index corresponding to the maximum label value is greater than 1, construct the word vectors of each word in the target text, and construct the word vectors of each category in the categories corresponding to the index corresponding to the maximum label value;

[0108] For any category in the categories corresponding to the index corresponding to the maximum label value, calculate the word vector angle and word vector distance between the word vector of this category and the word vectors of each word in the target text;

[0109] Calculate the mean value of the included angles of all word vectors corresponding to each category and the mean value of the distances of all word vectors, and perform the same-type feature transformation on the mean value of the included angles and the mean value of the distances corresponding to each category to obtain the target mean value;

[0110] According to the target mean value corresponding to each category, select the category with the smallest target mean value from all categories as the category of the target text.

[0111] It can be seen that when the number of categories corresponding to the index corresponding to the maximum label value is greater than 1, this optional embodiment provides multiple ways to accurately determine the category of the text by calculating the mean value of the included angles and the mean value of the distances between the word vectors of each category and the word vectors of all words in the text respectively, performing the same-type feature transformation on them, and then determining the category of the text from them, improving the flexibility of determining the category of the text.

[0112] In another optional embodiment, the method further includes the following operations:

[0113] Determine the sample text set corresponding to the sample industry object. The sample industry object has multiple levels. There are multiple categories in the lowest level among all levels, and the number of each category is greater than or equal to 1. The sample text set contains multiple sample texts;

[0114] Determine the target category to which each sample text belongs in the lowest-level hierarchy, and determine the total loss of each sample text based on the target category to which each sample text belongs, the number of all hierarchies, each category, and the determined label smoothing coefficient;

[0115] Based on the total loss of all sample texts, train the basic text classification model until the basic text classification model converges, obtain the converged basic text classification model, and determine the converged basic text classification model as the pre-trained text classification model.

[0116] In this optional embodiment, optionally, the label smoothing coefficient can be random or determined based on one or more of the type of the sample industry object, the number of hierarchies of the sample industry object, and the number of categories in the lowest-level hierarchy.

[0117] In this optional embodiment, the basic text classification model includes, but is not limited to, a text feature extractor constructed based on one or more of TextCNN, TextRNN, TextRNN_Att, BiLSTM, BiGRU that can identify text features in the transformer architecture.

[0118] It can be seen that in this optional embodiment, by calculating the total loss of each sample text through the number of hierarchies of the sample industry object, the categories in the lowest level, the label smoothing coefficient, and the type to which the sample text belongs in the lowest level, the calculation accuracy and reliability of the total loss can be improved, and the model is trained based on the total loss until the model converges, which can train an accurate text classification model, thereby further facilitating the improvement of text classification accuracy; and by training the basic text classification model with the total loss of the sample text calculated in combination with the label smoothing coefficient, the initial label value of the category to which the sample text belongs can be suppressed while enhancing the label values of other categories, reducing the occurrence of overfitting of the text classification model, thereby improving the training accuracy of the text classification model and further improving the generalization of the text classification model.

[0119] In another optional embodiment, determining the total loss of each sample text based on the target category to which each sample text belongs, the number of all hierarchies, each category, and the determined label smoothing coefficient includes:

[0120] For any sample text:

[0121] Determine the correlation between each category and the target category to which the sample text belongs;

[0122] Based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all hierarchies, determine the smoothed label value of each category;

[0123] Input the smoothed label values of each category into the basic text classification model and the loss layer for analysis, and obtain the propensity scores of each category corresponding to the sample text;

[0124] Determine the total loss of the sample text based on the propensity scores of each category corresponding to the sample text and the smoothed label values of that category.

[0125] In this optional embodiment, optionally, determining the correlation between each category and the target category to which the sample text belongs includes:

[0126] Analyze the hierarchical relationship between each category and the target category to which the sample text belongs, and determine the correlation between each category and the target category to which the sample text belongs according to the corresponding hierarchical relationship of each category.

[0127] In this optional embodiment, the calculation formula for the total loss of each sample text is as follows:

[0128]

[0129] In the formula, loss represents the total loss of each sample text, p j represents the smoothed label value of the jth category, and logq j represents the propensity score corresponding to the smoothed label value of the jth category, and k represents the total number of all categories in the lowest level.

[0130] It can be seen that in this optional embodiment, by determining the smoothed label values of each category in the lowest level based on the label smoothing coefficient and the determined correlation between each category and the category to which the sample text belongs, the determination accuracy and reliability of each smoothed label value can be improved, and it is output to the basic text classification model for analysis, and combined with the obtained propensity scores to jointly determine the total loss of the sample text, which can improve the determination accuracy and reliability of the total loss of the sample text, thus being beneficial to improving the training accuracy of the text classification model.

[0131] In another optional embodiment, based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all levels, determining the smoothed label value of each category corresponding to the sample text includes:

[0132] Based on the number of all levels and the correlation corresponding to each category, determine the label value allocation ratio of all categories, and based on the determined label smoothing coefficient, determine the initial label value of the index where the target category to which the sample text belongs is located;

[0133] According to the relevance corresponding to each category, all categories belonging to the same level as the target category to which the sample text belongs are determined from all categories, where the number of a certain level is greater than or equal to 1 and less than or equal to the number of all levels, and the value of a certain level is a positive integer;

[0134] Based on the label smoothing coefficient, the label value distribution ratio of all categories, and the number of all categories corresponding to each certain level, the smoothed label value of each category corresponding to each certain level is determined, and the initial label value corresponding to the sample text and the smoothed label values of each category corresponding to each certain level in all certain levels are determined as the smoothed label values of each category corresponding to the sample text.

[0135] In this optional embodiment, the lower the level of the common level to which the categories belong, the greater their relevance. The greater the relevance, the greater the label value distribution ratio. And the calculation method of the smoothed label value of each category at each level can be shown as follows:

[0136]

[0137] In the formula, target represents the smoothed label value of each category at the z-th level, is the label smoothing coefficient, N is the number of all levels of the sample industry object, N z represents the label value distribution ratio, P z represents the number of categories corresponding to the z-th level, Hz(j) represents obtaining the index of the j-th category at the z-th level, and Hz(i) represents obtaining the index of the target category i at the z-th level. For example, as Figure 6 shown, Figure 6 is an example schematic diagram of a text classification method based on label smoothing disclosed in an embodiment of the present invention. As Figure 6As shown, there are three levels, namely Level 1, Level 2, and Level 3. Among them, Level 1 is the lowest level and contains 10 categories, namely Category 1, Category 2, Category 3, Category 4, Category 5, Category 6, Category 7, Category 8, Category 9, and Category 10. Suppose the category of the sample text is Category 1. At this time, Category 1 and Category 2 belong to the same Level 1. Category 3 and Category 4, which also belong to Level 1, are not in the same level as Category 1 but belong to Level 2. Category 5, Category 6, and Category 7 are not in the same levels as Category 1 and Level 2 but belong to Level 3. Category 8, Category 9, and Category 10 do not belong to any level with Category 1. Therefore, the relevance of Category 2, (Category 3, Category 4), (Category 5, Category 6, Category 7), (Category 8, Category 9, and Category 10) to Category 1 decreases in turn, and the relevance of Category 8, Category 9, and Category 10 is 0. At this time, the smoothed label values can be determined for Category 2, (Category 3, Category 4), (Category 5, Category 6, Category 7) according to the smoothed label value distribution ratio of 3:2:1. Suppose the label smoothing coefficient is 0.1. Then the smoothed label value of Category 2 is [0.1*(3 / 6)] / 1 = 0.05, the smoothed label values of Category 3 and 4 are [0.1*(2 / 6)] / 2 ≈ 0.0167 respectively, and the smoothed label values of Category 5, 6, and 7 are [0.1*(1 / 6)] / 3 ≈ 0.0056 respectively. The smoothed label value of the sample industry object under this sample text is [0.9, 0.05, 0.0167, 0.0167, 0.0056, 0.0056, 0.0056, 0, 0, 0].

[0138] It can be seen that after determining the label value distribution ratio of each category and the initial label value of the category to which the sample text belongs, the smoothed label value of each category at this level is sequentially determined based on each category and the same level to which the category to which the sample text belongs belongs, which can accurately and efficiently determine the smoothed label value of each category, and further improve the accuracy and reliability of the determination of the total loss of the sample text.

[0139] In another alternative embodiment, the method may further include the following steps:

[0140] Determine the index position of the target category to which the sample text belongs, and determine the one-hot code label value of the sample text according to the index position of the target category to which the sample text belongs, the index position of each category in all categories except the target category, and the determined one-hot code strategy;

[0141] Among them, determining the smoothed label value of each category corresponding to each certain level based on the label smoothing coefficient, the label value distribution ratio of all categories, and the number of all categories corresponding to each certain level includes:

[0142] Determine the smoothed label value for each category corresponding to each certain level based on the one-hot label value of the sample text, the label smoothing coefficient, the label value allocation ratio for all categories, and the number of all categories corresponding to each certain level.

[0143] It can be seen that in this optional embodiment, by first determining the index position of the category to which the sample text belongs, the index positions of other categories in the lowest level, and the one-hot code strategy, the one-hot label value of the sample text is determined, and further combined with other parameters to determine the label smoothing value for each category at the lowest level, which can improve the accuracy and reliability of determining the label smoothing values for all categories, and thus is beneficial to further improving the accuracy of determining the total loss of the sample.

[0144] In another optional embodiment, after determining the correlation between each category and the target category to which the sample text belongs, the method may further include the following steps:

[0145] Determine the industry type of the sample industry object, and based on the industry type of the sample industry object and the correlation corresponding to each category, screen out all categories with a correlation greater than or equal to a preset correlation from all categories, and trigger the execution of the above operation of determining the smoothed label value for each category corresponding to the sample text based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all levels;

[0146] wherein each such category is a category with a correlation greater than or equal to the preset correlation, and the preset correlation is determined by the industry type of the sample industry object.

[0147] It can be seen that in this optional embodiment, after determining the correlation between each category and the category to which the sample text belongs, further removing the categories with relatively low correlations can reduce the amount of data participating in the calculation, improve the efficiency of determining the label smoothing value while ensuring accurate label smoothing values, and thus is beneficial to improving the efficiency of determining the total loss of the sample text.

[0148] Embodiment 2

[0149] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a method for training a classification model based on label smoothing disclosed in an embodiment of the present invention. Among them, Figure 2 the described method can be applied to a device for training a classification model based on label smoothing, such as a server, a cloud platform, etc., which is not limited in the embodiments of the present invention. As Figure 2 shown, the method for training a classification model based on label smoothing may include the following operations:

[0150] 201. Determine the sample text set corresponding to the sample industry object. The sample industry object has multiple levels. There are multiple categories in the lowest level among all levels, and the number of each category is greater than or equal to 1. The sample text set contains multiple sample texts.

[0151] 202. Determine the target category to which each sample text belongs in the lowest level, and determine the total loss of each sample text according to the target category to which each sample text belongs, the number of all levels, each category, and the determined label smoothing coefficient.

[0152] 203. Based on the total loss of all sample texts, train the basic text classification model until the basic text classification model converges, obtain the converged basic text classification model, and determine the converged basic text classification model as the pre-trained text classification model.

[0153] It should be noted that for the description of other related content, please refer to the above description of the related content and will not be elaborated here.

[0154] It can be seen that in the implementation Figure 2 The method described calculates the total loss of each sample text through the number of levels of the sample industry object, the categories in the lowest level, the label smoothing coefficient, and the type to which the sample text belongs in the lowest level, which can improve the calculation accuracy and reliability of the total loss, and trains the model based on the total loss until the model converges, and can train an accurate text classification model, which further helps to improve the text classification accuracy.

[0155] Embodiment III

[0156] Please refer to Figure 3 , Figure 3 which is a schematic structural diagram of a text classification device based on label smoothing disclosed in an embodiment of the present invention. As Figure 3 shown, the text classification device based on label smoothing may include:

[0157] A determination module 301, configured to determine the target industry object to which the target text of the text category to be recognized belongs. The target industry object has multiple target levels. There are multiple categories in the lowest target level among all target levels, and the number of each category is greater than or equal to 1;

[0158] An analysis module 302, configured to input the target text and all categories corresponding to the lowest target level of the target industry object into the pre-trained text classification model and a preset loss layer for analysis. The text classification model is a model trained with the sample texts whose corresponding label values are smoothed.

[0159] An acquisition module 303 is configured to acquire an analysis result output by a text classification model and a preset loss layer. The analysis result includes label values corresponding to indexes of each category in the target level with the lowest level for the target text.

[0160] A determination module 301 is further configured to determine, according to the label value corresponding to each category, the category corresponding to the index corresponding to the maximum label value as the category of the target text.

[0161] It can be seen that implementing Figure 3 the described device can analyze by inputting the text to be classified and all categories in the lowest level of the hierarchical objects of the industry to which the text belongs into a text classification model trained with label-smoothed sample texts, and determine the category corresponding to the index corresponding to the maximum label value analyzed as the category of the text, which can improve the analysis accuracy of the specific category to which the text belongs, obtain accurate multi-level labels, so that the classified labels can accurately express the text meaning and facilitate the classification and induction of the text, and further contribute to improving the availability of the text.

[0162] In an optional embodiment, the determination module 301 is further configured to determine a set of sample texts corresponding to a sample industry object. The sample industry object has multiple levels. There are multiple categories in the lowest level among all levels, and the number of each category is greater than or equal to 1. The set of sample texts includes multiple sample texts.

[0163] The determination module 301 is further configured to determine the target category to which each sample text belongs in the lowest level.

[0164] The determination module 301 is further configured to determine the total loss of each sample text according to the target category to which each sample text belongs, the number of all levels, each category, and the determined label smoothing coefficient.

[0165] And, as Figure 4 shown, the device further includes:

[0166] A training module 304 is configured to train a basic text classification model based on the total loss of all sample texts until the basic text classification model converges, and obtain a converged basic text classification model.

[0167] The determination module 301 is further configured to determine the converged basic text classification model as the pre-trained text classification model.

[0168] It can be seen that implementing Figure 4The described device calculates the total loss of each sample text through the number of levels of the sample industry object, the categories at the lowest level, the label smoothing coefficient, and the type to which the sample text belongs at the lowest level, which can improve the accuracy and reliability of the calculation of the total loss, and trains the model based on the total loss until the model converges, and can train an accurate text classification model, thereby further facilitating the improvement of the text classification accuracy.

[0169] In yet another optional embodiment, the specific manner in which the determination module 301 determines the total loss of each sample text according to the target category to which each sample text belongs, the number of all levels, each category, and the determined label smoothing coefficient includes:

[0170] For any sample text:

[0171] Determine the correlation between each category and the target category to which the sample text belongs;

[0172] Based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all levels, determine the smoothed label value of each category;

[0173] Input the smoothed label value of each category into the basic text classification model and the loss layer for analysis to obtain the propensity score of each category corresponding to the sample text;

[0174] Based on the propensity score of each category corresponding to the sample text and the smoothed label value of this category, determine the total loss of the sample text.

[0175] In this optional embodiment, the specific manner in which the determination module 301 determines the correlation between each category and the target category to which the sample text belongs includes:

[0176] Analyze the hierarchical relationship between each category and the target category to which the sample text belongs, and determine the correlation between each category and the target category to which the sample text belongs according to the hierarchical relationship corresponding to each category.

[0177] In this optional embodiment, the calculation formula for the total loss of each sample text is as follows:

[0178]

[0179] In the formula, loss represents the total loss of each sample text, p j represents the smoothed label value of the jth category, logq j represents the propensity score corresponding to the smoothed label value of the jth category, and k represents the total number of all categories in the lowest level.

[0180] It can be seen that the implementation Figure 4The described device can also determine the smoothed label values of each category in the lowest level based on the label smoothing coefficient and the correlation between each category in the determined lowest level and the category to which the sample text belongs, which can improve the accuracy and reliability of the determination of each smoothed label value, output it to the basic text classification model for analysis, and jointly determine the total loss of the sample text in combination with the propensity score obtained from the analysis, which can improve the accuracy and reliability of the determination of the total loss of the sample text, thereby facilitating the improvement of the training accuracy of the text classification model.

[0181] In yet another alternative embodiment, the specific manner in which the determination module 301 determines the smoothed label values of each category corresponding to the sample text based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all levels includes:

[0182] Based on the number of all levels and the correlation corresponding to each category, determine the label value allocation ratio of all categories, and based on the determined label smoothing coefficient, determine the initial label value of the index where the target category to which the sample text belongs is located;

[0183] According to the correlation corresponding to each category, determine all categories that belong to the same level as the target category to which the sample text belongs from all categories, where the number of a certain level is greater than or equal to 1 and less than or equal to the number of all levels, and the value of a certain level is a positive integer;

[0184] Based on the label smoothing coefficient, the label value allocation ratio of all categories, and the number of all categories corresponding to each certain level, determine the smoothed label value of each category corresponding to each certain level, and determine the initial label value corresponding to the sample text and the smoothed label values of each category corresponding to each certain level in all certain levels as the smoothed label values of each category corresponding to the sample text.

[0185] The calculation method of the smoothed label value of each category of each level can be shown as follows:

[0186]

[0187] In the formula, target represents the smoothed label value of each category at the z-th level, is the label smoothing coefficient, N is the number of all levels of the sample industry object, N z represents the label value allocation ratio, P z represents the number of categories corresponding to the z-th level, Hz(j) represents obtaining the index of the j-th category at the z-th level, and Hz(i) represents obtaining the index of the target category i at the z-th level.

[0188] It can be seen that the implementation Figure 4The described device is also capable of, after determining the label value allocation ratio for each category and the initial label value of the sample text's belonging category, successively determining the smoothed label value of each category at the same level as the sample text's belonging category based on each category, which can accurately and efficiently determine the smoothed label value of each category, thereby further improving the accuracy and reliability of determining the total loss of the sample text.

[0189] In yet another alternative embodiment, the determination module 301 is further configured to determine the index position of the target category to which the sample text belongs, and determine the one-hot code label value of the sample text according to the index position of the target category to which the sample text belongs, the index position of each category in all categories except the target category, and the determined one-hot code strategy;

[0190] Wherein, the specific manner in which the determination module 301 determines the smoothed label value of each category corresponding to each certain level based on the label smoothing coefficient, the label value allocation ratio of all categories, and the number of all categories corresponding to each certain level includes:

[0191] Based on the one-hot code label value of the sample text, the label smoothing coefficient, the label value allocation ratio of all categories, and the number of all categories corresponding to each certain level, determine the smoothed label value of each category corresponding to each certain level.

[0192] It can be seen that implementing Figure 4 The described device is also capable of first determining the index position of the category to which the sample text belongs, the index positions of other categories in the lowest level, and the one-hot code strategy, determining the one-hot code label value of the sample text, and further combining it with other parameters to determine the label smoothing value of each category at the lowest level, which can improve the accuracy and reliability of determining the label smoothing value of all categories, and thus is conducive to further improving the accuracy of determining the total loss of the sample stupid.

[0193] In yet another alternative embodiment, the determination module 301 is further configured to determine the industry type of the sample industry object after determining the correlation between each category and the target category to which the sample text belongs;

[0194] And, as Figure 4 shown, the device further includes:

[0195] A screening module 305, configured to screen all categories with a correlation greater than or equal to a preset correlation from all categories according to the industry type of the sample industry object and the correlation corresponding to each category, and trigger the determination module 301 to perform the above operation of determining the smoothed label value of each category corresponding to the sample text based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all levels;

[0196] Among them, each of these categories is a category with a relevance greater than or equal to a preset relevance, and the preset relevance is determined by the industry type of the sample industry object.

[0197] It can be seen that when implementing Figure 4 the described device can also further remove categories with relatively low relevance after determining the relevance between each category and the category to which the sample text belongs, which can reduce the amount of data participating in the calculation. While ensuring an accurate label smoothing value, it can improve the determination efficiency of the label smoothing value, thereby facilitating the improvement of the determination efficiency of the total loss of the sample text.

[0198] In an alternative embodiment, the device may further include:

[0199] A construction module 306, configured to construct word vectors for each word in the target text and construct word vectors for each category in the category corresponding to the index to which the maximum label value belongs when the number of categories corresponding to the index to which the maximum label value belongs is greater than 1;

[0200] A calculation module 307, configured to calculate the word vector angle and word vector distance between the word vector of any category in the category corresponding to the index to which the maximum label value belongs and the word vectors of each word in the target text;

[0201] The calculation module 307 is further configured to calculate the angle mean of all word vector angles and the distance mean of all word vector distances corresponding to each category, and perform the same-type feature transformation on the angle mean and distance mean corresponding to each category to obtain a target mean;

[0202] A screening module 305 is further configured to screen, from all categories, the category with the minimum target mean as the category of the target text according to the target mean corresponding to each category.

[0203] It can be seen that when implementing Figure 4 the described device can also, when the number of categories corresponding to the index to which the maximum label value belongs is greater than 1, determine the category of the text by separately calculating the angle mean and distance mean corresponding to the word vectors of each category and the word vectors of all words in the text, performing the same-type feature transformation on them, and then determining the category of the text according to the transformed mean, providing multiple ways to accurately determine the category of the text and improving the flexibility in determining the category of the text.

[0204] Embodiment 4

[0205] Please refer to Figure 5 , Figure 5 which is a schematic structural diagram of a classification model training device based on label smoothing disclosed in an embodiment of the present invention. As Figure 5 shown, the classification model training device based on label smoothing may include:

[0206] A determination module 301, configured to determine a set of sample texts corresponding to a sample industry object. The sample industry object has multiple levels, and there are multiple categories in the lowest level among all levels, and the number of each category is greater than or equal to 1. The set of sample texts includes multiple sample texts;

[0207] The determination module 301 is further configured to determine the target category to which each sample text belongs in the lowest level;

[0208] The determination module 301 is further configured to determine the total loss of each sample text according to the target category to which each sample text belongs, the number of all levels, each category, and the determined label smoothing coefficient;

[0209] A training module 302, configured to train a basic text classification model based on the total loss of all sample texts until the basic text classification model converges, and obtain a converged basic text classification model;

[0210] The determination module 301 is further configured to determine the converged basic text classification model as a pre-trained text classification model.

[0211] It can be seen that the Figure 5 described device calculates the total loss of each sample text through the number of levels of the sample industry object, the categories of the lowest level, the label smoothing coefficient, and the type to which the sample text belongs in the lowest level, which can improve the calculation accuracy and reliability of the total loss, and trains the model based on the total loss until the model converges, and can train an accurate text classification model, thereby further facilitating the improvement of the text classification accuracy; and by training the basic text classification model with the total loss of the sample text calculated in combination with the label smoothing coefficient, it can suppress the initial label value of the category to which the sample text belongs and enhance the label values of other categories, reduce the occurrence of overfitting of the text classification model, thereby improving the training accuracy of the text classification model, and further improving the generalization of the text classification model.

[0212] Embodiment 5

[0213] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of another text processing device based on label smoothing disclosed in an embodiment of the present invention. The text processing device based on label smoothing includes a text classification device based on label smoothing or a classification model training device based on label smoothing. Among them, as Figure 6 shown, the device may include:

[0214] A memory 501 storing executable program code;

[0215] A processor 502 coupled to the memory 501;

[0216] Further, it may further include an input interface 503 and an output interface 504 coupled to the processor 502;

[0217] Wherein, the processor 502 calls the executable program code stored in the memory 501 to execute some or all of the steps of the text processing method based on label smoothing disclosed in Embodiment 1 or Embodiment 2 of the present invention. The text processing method based on label smoothing includes a text classification method based on label smoothing or a classification model training method based on label smoothing.

[0218] Embodiment Six

[0219] An embodiment of the present invention discloses a computer storage medium. The computer storage medium stores computer instructions, which are used to execute some or all of the steps of the text classification method based on label smoothing disclosed in Embodiment 1 of the present invention when the computer instructions are called.

[0220] Embodiment Seven

[0221] An embodiment of the present invention discloses a computer storage medium. The computer storage medium stores computer instructions, which are used to execute some or all of the steps of the classification model training method based on label smoothing disclosed in Embodiment 2 of the present invention when the computer instructions are called.

[0222] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0223] Through the specific descriptions of the above embodiments, those skilled in the art can clearly understand that each implementation can be realized by means of software plus a necessary general hardware platform. Of course, it can also be realized by hardware. Based on such an understanding, the above technical solution, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, and the storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other computer-readable medium that can be used to carry or store data.

[0224] Finally, it should be noted that: what is disclosed in an embodiment of the present invention, a method and device for text classification and text classification model training based on label smoothing, is only a preferred embodiment of the present invention, and is only used to illustrate the technical solution of the present invention, rather than limiting it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A text classification method based on label smoothing, characterized in that, The method includes: Determining a target industry object to which a target text belonging to a category of text to be recognized belongs, where the target industry object has multiple target levels, there are multiple categories in the target level with the lowest level among all the target levels, and the number of each category is greater than or equal to 1; Inputting the target text and all the categories existing in the target level with the lowest level in the target industry object into a pre-trained text classification model and a preset loss layer for analysis, where the text classification model is a model trained with sample texts whose corresponding label values are smoothed; Obtaining an analysis result output by the text classification model and the preset loss layer, where the analysis result includes label values corresponding to indices of each category to which the target text belongs in the target level with the lowest level, and determining, according to the label value corresponding to each category, the category corresponding to the index corresponding to the maximum label value as the category of the target text; Wherein, the text classification model is obtained by performing a training operation on a basic text classification model using the total loss of all sample texts in a sample text set corresponding to a sample industry object; Wherein, for any one of the sample texts, the total loss of the sample text is determined in the following manner: Determining the correlation between each category of the sample text and the target category to which the sample text belongs in the lowest level; Based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all the levels, determining the smoothed label value of each category; Inputting the smoothed label value of each category into the basic text classification model and the loss layer for analysis to obtain the propensity score of each category; Determining the total loss of the sample text based on the propensity score of each category and the smoothed label value of the category; The calculation formula of the total loss of the sample text is as follows: wherein, the loss represents the total loss of the sample text, and p j represents the smoothed label value of the j-th category, and logq j represents the propensity score corresponding to the smoothed label value of the j-th category, and k represents the total number of all categories in the lowest level; The calculation method of the smoothed label value of each category of each level can be shown as follows: wherein, the target represents the smoothed label value of each category at the z-th level, the is the label smoothing coefficient, the N is the number of all levels of the sample industry object, the N z represents the label value allocation ratio, the P z represents the number of corresponding categories at the z-th level, the Hz(j) represents obtaining the index of the j-th category at the z-th level, and the Hz(i) represents obtaining the index of the target category i at the z-th level.

2. The text classification method based on label smoothing according to claim 1, characterized in that, The method further includes: Determining a sample text set corresponding to the sample industry object, where the sample industry object has multiple levels, there are multiple categories in the level with the lowest level among all the levels, and the number of each category is greater than or equal to 1, and the sample text set includes multiple sample texts; Determining the target category to which each sample text belongs in the lowest level, and determining the total loss of each sample text according to the target category to which each sample text belongs, the number of all the levels, each category, and the determined label smoothing coefficient; Training the basic text classification model based on the total loss of all the sample texts until the basic text classification model converges, obtaining the converged basic text classification model, and determining the converged basic text classification model as the pre-trained text classification model.

3. The text classification method based on label smoothing according to claim 1 or 2, characterized in that, The determining the smoothed label value of each category based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all the levels includes: Based on the number of all the said levels and the relevance corresponding to each said category, determine the label value allocation ratio of all the said categories, and based on the determined label smoothing coefficient, determine the initial label value of the index of the target category to which the said sample text belongs; According to the relevance corresponding to each said category, determine, from all the said categories, all the categories that belong to the same level as the target category to which the said sample text belongs, where the number of the said level is greater than or equal to 1 and less than or equal to the number of all the said levels, and the value of the said level is a positive integer; Based on the label smoothing coefficient, the label value allocation ratio of all the said categories, and the number of all the said categories corresponding to each said level, determine the smoothed label value of each said category corresponding to each said level, and determine the initial label value corresponding to the said sample text and the smoothed label value of each said category corresponding to each said level in all the said levels as the smoothed label value of each said category.

4. The text classification method based on label smoothing according to claim 3, characterized in that, The method further includes: Determine the index position of the target category to which the said sample text belongs, and according to the index position of the target category to which the said sample text belongs, the index position of each said category in all the said categories except the target category, and the determined one-hot code strategy, determine the one-hot code label value of the said sample text; Among them, the determining the smoothed label value of each said category corresponding to each said level based on the label smoothing coefficient, the label value allocation ratio of all the said categories, and the number of all the said categories corresponding to each said level includes: Based on the one-hot code label value of the said sample text, the label smoothing coefficient, the label value allocation ratio of all the said categories, and the number of all the said categories corresponding to each said level, determine the smoothed label value of each said category corresponding to each said level.

5. The text classification method based on label smoothing according to any one of claims 1, 2 and 4, characterized in that, After determining the relevance between each said category of the said sample text and the target category to which the said sample text belongs in the lowest-level hierarchy, the method further includes: Determine the industry type of the said sample industry object, and according to the industry type of the said sample industry object and the relevance corresponding to each said category, screen, from all the said categories, all the said categories whose relevance is greater than or equal to a preset relevance, and trigger the execution of the operation of determining the smoothed label value of each said category based on the determined label smoothing coefficient, the relevance corresponding to each said category, and the number of all the said levels; Among them, each such category is a category whose relevance is greater than or equal to the preset relevance, and the preset relevance is determined by the industry type of the said sample industry object.

6. The text classification method based on label smoothing according to any one of claims 1, 2, and 4, characterized in that The determining the relevance between each said category of the said sample text and the target category to which the said sample text belongs in the lowest-level hierarchy includes: Analyze the hierarchical relationship between each said category and the target category to which the said sample text belongs, and according to the hierarchical relationship corresponding to each said category, determine the relevance between each said category and the target category to which the said sample text belongs.

7. A classification model training method based on label smoothing, characterized in that The method includes: Determine a set of sample texts corresponding to a sample industry object. The sample industry object has multiple levels. Among all the levels, the lowest level has multiple categories, and the number of each category is greater than or equal to 1. The set of sample texts includes multiple sample texts; Determine the target category to which each sample text belongs in the lowest level, and determine the total loss of each sample text according to the target category to which each sample text belongs, the number of all levels, each category, and the determined label smoothing coefficient; Based on the total losses of all the sample texts, train a basic text classification model until the basic text classification model converges, obtain the converged basic text classification model, and determine the converged basic text classification model as the pre-trained text classification model; Among them, for any sample text, the total loss of the sample text is determined in the following manner: Determine the correlation between each category of the sample text and the target category to which the sample text belongs in the lowest level; Based on the determined label smoothing coefficient, the correlation corresponding to each category, and the number of all levels, determine the smoothed label value of each category; Input the smoothed label value of each category into the basic text classification model and the loss layer for analysis to obtain the propensity score of each category; Based on the propensity score of each category corresponding to the sample text and the smoothed label value of the category, determine the total loss of the sample text; The calculation formula for the total loss of the sample text is as follows: wherein, the loss represents the total loss of the sample text, and p j represents the smoothed label value of the j-th category, and logq j represents the propensity score corresponding to the smoothed label value of the j-th category, and k represents the total number of all categories in the lowest level; The calculation method for the smoothed label value of each category of each level is as follows: Wherein, the target represents the smoothed label value of each category at the z-th level, and the is the label smoothing coefficient, N is the number of all levels of the sample industry object, and the N z represents the label value allocation ratio, and the P z represents the number of corresponding categories at the z-th level, Hz(j) represents obtaining the index of the j-th category at the z-th level, and Hz(i) represents obtaining the index of the target category i at the z-th level.

8. A text classification device based on label smoothing, characterized in that The device is used to execute the text classification method based on label smoothing according to any one of claims 1-6, and the device includes: A determination module, configured to determine the target industry object to which the target text of the text category to be recognized belongs. The target industry object has multiple target levels. Among all the target levels, the lowest target level has multiple categories, and the number of each category is greater than or equal to 1; An analysis module, configured to input the target text and all the categories existing in the lowest target level of the target industry object into a pre-trained text classification model and a preset loss layer for analysis. The text classification model is a model trained with sample texts whose corresponding label values are smoothed; An acquisition module, configured to acquire the analysis result output by the text classification model and the preset loss layer. The analysis result includes the label value corresponding to the index of each category to which the target text belongs in the lowest target level; The determination module is further configured to determine, according to the label value corresponding to each category, the category corresponding to the index corresponding to the maximum label value as the category of the target text.

9. A text classification device based on label smoothing, characterized in that The device includes: A memory storing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory and executes the text classification method based on label smoothing according to any one of claims 1-6.

Images