Text classification model training method and device, text classification method and device

Abstract

The application discloses a text classification model training method and device and a text classification method and device. The text classification model training method comprises the following steps: determining a training text set, wherein the training text set comprises no-labeled rule violation texts, rule violation texts with labeled prohibited categories, and normal texts; generating training data by using hidden characters and target texts; inputting the training data into a text classification model to obtain target characters predicted by the text classification model and classification results predicted by the text classification model based on the training data; and adjusting parameters of the text classification model based on a text semantic loss value and a classification loss value until the text semantic loss value and the classification loss value meet preset conditions, so that a trained text classification model is obtained. It can be seen that the text classification model trained by the text classification model training method provided by the application has high semantic analysis capability and rule violation text distinguishing capability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information processing, and more particularly to a text classification model training method and device, and a text classification method and device. BACKGROUND

[0002] With the rapid development of Internet technology, reading text information on the Internet has become a common way of leisure and entertainment. However, with the lowering of the threshold for publishing text information on the Internet, there are a large number of illegal texts on the Internet that are not suitable for display to users, especially minors. Therefore, it is necessary to process the illegal texts to protect the physical and mental health of users, especially minors. The prerequisite for processing illegal texts is to find illegal texts from a large number of text information.

[0003] Therefore, in order to find illegal texts from a large number of text information, a classification model capable of identifying illegal texts can be introduced to classify each text information. SUMMARY

[0004] Therefore, the present application provides a text classification model training method and device, and a text classification method and device, which are used to train a classification model capable of identifying illegal texts to classify each text information.

[0005] In order to achieve the above purpose, the present scheme is as follows:

[0006] A text classification model training method comprises:

[0007] determining a training text set, wherein the training text set contains a plurality of unlabeled illegal texts, a plurality of illegal texts with labeled prohibited categories, and a plurality of unlabeled normal texts;

[0008] selecting a target text from the training text set in turn;

[0009] generating training data by using a preset hidden character and the target text, wherein the training data is the target text with some characters replaced by the hidden character;

[0010] inputting the training data into a text classification model to obtain a target character predicted by the text classification model and a classification result predicted by the text classification model based on the training data, wherein the classification result is a binary classification result corresponding to a plurality of prohibited categories, and the text classification model is a text classification model to be trained;

[0011] calculating a text semantic loss value of the text classification model according to the target character and the target text;

[0012] According to the classification result and the target text, a classification loss value of the text classification model is calculated;

[0013] Based on the text semantic loss value and the classification loss value, parameters of the text classification model are adjusted until the text semantic loss value and the classification loss value meet a preset condition, so as to obtain a trained text classification model.

[0014] Optionally, the method further comprises:

[0015] The bias parameter and the weight parameter of the trained text classification model are adjusted to obtain a processed text classification model, and an output of the processed text classification model is a classification result corresponding to an input text.

[0016] Optionally, the bias parameter and the weight parameter of the trained text classification model are adjusted to obtain a processed text classification model, and the method comprises:

[0017] The bias parameter and the weight parameter of the trained text classification model are adjusted to obtain a processed text classification model.

[0018] Optionally, the bias parameter and the weight parameter of the trained text classification model are adjusted to obtain a processed text classification model, and the method comprises:

[0019] The bias parameter and the weight parameter of the trained text classification model are adjusted to obtain a processed text classification model.

[0020] The adjustment formula is as follows:

[0021] H = BERT (X; θ)

[0022]

[0023]

[0024]

[0025]

[0026]

[0027] Wherein, X represents the input of the trained text classification model, θ represents the weight parameter of the trained text classification model, BERT represents semantic coding using a text classification model, H represents a semantic vector coded by a text classification model, Slice1 represents intercepting a semantic vector coded by a text classification model, represents a classification result corresponding semantic vector, represents a semantic vector after full connection, LN represents a Layer Normalization normalization operation, GELU represents a Gaussian Error Linear Unit activation function, W1 represents a weight parameter of a full connection layer, and B1 represents a bias parameter of a full connection layer, represents a vector matrix containing only a target character, Slice2 represents an intercepted vector matrix of a target character in a dictionary, and ETable represents a vector matrix of a dictionary. represents a score of a binary classification result corresponding to a prohibited category, represents a transpose of a vector matrix containing only a target character, and B2 represents a bias parameter of dimension transformation (dimension increase), represents a probability of a binary classification result corresponding to a prohibited category, and Softmax represents a probability obtained by using a softmax function, and the dictionary is a pre-established character database.

[0028] Optionally, training data is generated by using a preset hidden character and the target text, including:

[0029] The hidden character is used to replace part of characters of the target text to obtain a replacement text.

[0030] If the target text does not exist a label, the replacement text is directly used as the training data.

[0031] If the target text exists a label, the replacement text is processed by using a preset text template to obtain training data.

[0032] Optionally, the text template includes a fixed order and a specific position corresponding to a plurality of prohibited categories, and further includes a specific position of the replacement text.

[0033] The replacement text is processed by using the preset text template to obtain training data, including:

[0034] An annotation label of the target text corresponding to the replacement text is used to determine a binary classification result corresponding to each prohibited category.

[0035] Based on the fixed order of the plurality of prohibited categories in the text template, an order of each binary classification result is determined.

[0036] Based on the order of the various binary classification results, a combination of binary classification results is formed;

[0037] Training data is generated based on the specific locations corresponding to multiple prohibited categories in the text template, the specific locations of the replacement text in the text template, the combination of the binary classification results, and the replacement text.

[0038] Optionally, the step of generating training data based on the specific locations corresponding to multiple prohibited categories in the text template, the specific locations of replacement texts in the text template, the combination of binary classification results, and the replacement texts includes:

[0039] Based on the specific locations corresponding to multiple prohibited categories in the text template, as well as the specific locations of the replacement text, the binary classification results and the replacement text are combined to obtain combined data;

[0040] A preset prefix character is added to the combined data, and a suffix character is added between the binary classification result of the combined data and the replacement text to obtain training data. When the training data is input into the text classification model, the text classification model distinguishes and identifies the binary classification result combination and the replacement text based on the prefix character and the suffix character.

[0041] A text classification model training device, comprising:

[0042] The determining unit is used to determine a training text set, which includes multiple unlabeled violation texts, multiple violation texts labeled with prohibited categories, and multiple unlabeled normal texts.

[0043] A selection unit is used to sequentially select target text from the training text set;

[0044] The generation unit is used to generate training data using preset hidden characters and the target text, wherein the training data is the target text in which some characters are replaced by hidden characters;

[0045] A classification unit is used to input the training data into a text classification model to obtain the target character predicted by the text classification model and the classification result predicted by the text classification model based on the training data. The classification result is a binary classification result corresponding to multiple prohibited categories. The text classification model is a text classification model to be trained.

[0046] The calculation unit is used to calculate the text semantic loss value of the text classification model based on the target character and the target text.

[0047] The unit is used to calculate the classification loss value of the text classification model based on the classification result and the target text.

[0048] The adjustment unit is used to adjust the parameters of the text classification model based on the text semantic loss value and the classification loss value until the text semantic loss value and the classification loss value meet the preset conditions, thereby obtaining the trained text classification model.

[0049] A text classification method, comprising:

[0050] Obtain the text information to be classified;

[0051] The text classification model trained using the above-mentioned text classification model training method is used to classify the text information to be classified, and the classification result is obtained. The classification result includes binary classification results corresponding to multiple prohibited categories.

[0052] A text classification device, comprising:

[0053] The text acquisition unit is used to acquire the text information to be classified.

[0054] The information classification unit is used to classify the text information to be classified by the text classification model trained by the above text classification model training method, and to obtain the classification result, which includes binary classification results corresponding to multiple prohibited categories.

[0055] As can be seen from the above technical solution, the text classification model training method provided in this application determines a training text set, which includes multiple unlabeled violation texts, multiple violation texts labeled with prohibited categories, and multiple unlabeled normal texts; selects target texts sequentially from the training text set; generates training data using preset hidden characters and the target texts, wherein the training data is target texts in which some characters are replaced by hidden characters; inputs the training data into a text classification model to obtain the target characters predicted by the text classification model and the classification results predicted by the text classification model based on the training data, wherein the classification results are binary classification results corresponding to multiple prohibited categories; Through the above process, the semantic analysis capability of the text classification model can be trained by hiding some characters, and a mixed training method using normal text and illegal text can be adopted to enable the text classification model to learn the difference between normal text and illegal samples, thereby improving the discrimination ability of the text classification model. Based on the target character and the target text, the text semantic loss value of the text classification model is calculated; based on the classification result and the target text, the classification loss value of the text classification model is calculated; based on the text semantic loss value and the classification loss value, the parameters of the text classification model are adjusted until the text semantic loss value and the classification loss value meet preset conditions, thus obtaining the trained text classification model. Through the above process, the parameters of the text classification model can be adjusted using the loss value to improve the semantic analysis capability and illegal text recognition capability of the text classification model. It is evident that the text classification model trained using the text classification model training method provided in this application possesses high semantic analysis capability and the ability to distinguish illegal text. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0057] Figure 1 This is a flowchart of a text classification model training method disclosed in this application;

[0058] Figure 2 This is a structural block diagram of a text classification model training device disclosed in an embodiment of this application;

[0059] Figure 3 This is a hardware structure block diagram of a text classification model training device disclosed in an embodiment of this application. Detailed Implementation

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0061] The text classification model training method provided in this application can be applied to a wide range of general-purpose or special-purpose computing device environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, and distributed computing environments including any of the above devices.

[0062] Next, combine Figure 1 The text classification model training method of this application is described in detail, including the following steps:

[0063] Step S1: Determine the training text set.

[0064] Specifically, the training text set may contain multiple unlabeled violation texts, multiple violation texts labeled with prohibited categories, and multiple unlabeled normal texts.

[0065] The ratio of unlabeled violation texts, violation texts labeled with prohibited categories, and unlabeled normal texts in the training text set can be set according to actual training needs. Generally speaking, violation texts labeled with prohibited categories can be the main component in the training text set.

[0066] The training text set can contain violation texts corresponding to multiple prohibited categories, and the same violation text can correspond to one or more prohibited categories. The same prohibited category can correspond to multiple violation texts.

[0067] The prohibited categories can be set according to actual needs.

[0068] Step S2: Select target texts sequentially from the training text set.

[0069] Specifically, target texts can be selected sequentially from the training text set. The target texts can be unlabeled violation texts, violation texts labeled with prohibited categories, or normal texts without labels.

[0070] Step S3: Using preset hidden characters and the target text, generate training data, wherein the training data is the target text in which some characters are replaced by hidden characters.

[0071] Specifically, some characters in the selected target text can be hidden. Furthermore, some words in the target text can be hidden using hidden characters, or some words in the target text can be replaced by randomly selected characters to achieve the purpose of hiding.

[0072] Training data can be generated in various ways using preset hidden characters and the target text. For example, some words in the target text can be randomly replaced to generate training data; or sensitive words in the target text can be identified and replaced with hidden characters to generate training data.

[0073] Among them, the characters replaced by hidden characters in the target text can account for 15% of the total characters in the target text.

[0074] Step S4: Input the training data into the text classification model to obtain the target character predicted by the text classification model and the classification result predicted by the text classification model based on the training data.

[0075] Specifically, the text classification model can be the text classification model that needs to be trained.

[0076] Training data can be input into a text classification model for training, resulting in the target characters and classification results output by the text classification model.

[0077] The text classification model obtains the classification results based on the training data and the target characters predicted by the training data.

[0078] The target character can be the character in the target text that is masked by hidden characters, as predicted by the text classification model based on the training data.

[0079] The classification result is a binary classification result corresponding to multiple prohibited categories. For example, multiple prohibited categories can be a first prohibited type and a second prohibited type. The binary classification result is "yes" or "no". Then the classification result can be "yes" for the first prohibited type and "no" for the second prohibited type.

[0080] If the target text corresponding to the training data is normal text, the binary classification result corresponding to the prohibited category will be negative.

[0081] Step S5: Calculate the text semantic loss value of the text classification model based on the target character and the target text.

[0082] Specifically, the target character can be replaced with the hidden character in the training data to obtain the predicted text; the cross-entropy loss between the predicted text and the target text can be calculated, and based on this semantic distance, the text semantic loss value of the text classification model can be calculated.

[0083] Step S6: Calculate the classification loss value of the text classification model based on the classification result and the target text.

[0084] Specifically, it is determined whether the target text has a label. If a label exists, the classification loss value of the text classification model is calculated based on the distance between the classification result and the label. If no label exists, it is determined whether the target text is normal text. If it is normal text and the classification result shows that there is no matching prohibited category in the training data, the classification loss value is determined to be 0. If the target text is illegal text and no label exists, the classification result shows that there is a matching prohibited category in the training data, and the classification loss value is directly determined to be 0.

[0085] Step S7: Based on the text semantic loss value and the classification loss value, adjust the parameters of the text classification model until the text semantic loss value and the classification loss value meet the preset conditions, and obtain the trained text classification model.

[0086] Specifically, the parameters of the text classification model can be adjusted based on the magnitude of the text semantic loss value and the classification loss value until the text semantic loss value and the classification loss value are both less than a threshold.

[0087] The threshold can be set in advance according to actual needs; different thresholds can be used for different accuracy requirements.

[0088] Alternatively, a loss function can be used to calculate the loss during the training process of the text classification model, utilizing the training data, target characters, and classification results. The loss function is shown below:

[0089]

[0090] Where i represents the i-th character in the training data; j represents the j-th character in the dictionary, from which the target character is selected, and the dictionary can include 21128 words; L represents the loss value corresponding to a single training data point; N represents the number of characters replaced by the hidden character in the training data; and m... i Indicates whether the i-th character in the training data is replaced by the hidden character, y j Indicates whether the j-th character in the training data is a label, p j This represents the probability that the j-th character in the dictionary is the target character.

[0091] As can be seen from the above technical solutions, this application, through its text classification model training method, can train the semantic analysis capability of the text classification model by hiding some characters, and employs a mixed training method using normal text and illegal text to enable the text classification model to learn the differences between normal text and illegal samples, thereby improving the discrimination ability of the text classification model. The parameters of the text classification model can be adjusted using the loss value to achieve the goal of improving the semantic analysis capability and the ability to identify illegal text. Therefore, the text classification model trained using the method provided in this application possesses high semantic analysis capability and the ability to identify illegal text.

[0092] Furthermore, by training the text classification model with a mixture of unlabeled and labeled prohibited texts, as well as unlabeled normal texts, supervised and self-supervised learning of the text classification model can be achieved. That is, semi-supervised learning can be carried out by making full use of labeled and unlabeled texts, thereby further improving the reliability and learning ability of the text classification model.

[0093] In some embodiments of this application, considering that the trained text classification model can predict target characters, but in the actual prediction process, the text classification model does not need to predict characters, only the classification result, therefore, after training the text classification model, a processing procedure for the text classification model can be added, and the functions of the text classification model can be reduced, in order to accelerate the prediction of classification results. The following is a detailed description of the processing procedure, the steps of which are as follows:

[0094] S8. Adjust the bias parameters and weight parameters of the trained text classification model to obtain the processed text classification model. The output of the processed text classification model is the classification result corresponding to the input text.

[0095] Specifically, the parameters of the text classification model can be adjusted so that the output of the text classification model is only the classification result, which includes binary classification results corresponding to multiple prohibited categories.

[0096] The parameters that can be adjusted can be bias parameters and weight parameters.

[0097] As can be seen from the above technical solution, compared with the previous embodiment, this embodiment adds the process of adjusting the bias parameters and weight parameters of the text classification model. Through the above process, the process of the text classification model predicting target characters can be reduced. When the text classification model only outputs the classification result, the prediction difficulty of the text classification model can be further reduced and the prediction process can be reduced, thereby accelerating the prediction of the text classification model and improving the prediction efficiency of the text classification model.

[0098] In some embodiments of this application, the process of adjusting the bias parameters and weight parameters of the trained text classification model to obtain the processed text classification model is described in detail below:

[0099] S80. Adjust the weight parameters and bias parameters related to the predicted target character and the predicted classification result in the trained text classification model to obtain the processed text classification model.

[0100] Specifically, when adjusting the bias and weight parameters of the text classification model, only the parameters related to the predicted target character and the predicted classification result can be adjusted.

[0101] As can be seen from the above technical solution, this embodiment provides an optional way to adjust the parameters of a text classification model. It can adjust the parameters related to the predicted target character and the predicted classification result in the text classification model, thereby further improving the efficiency of adjusting the text classification model and further improving the discrimination efficiency of the text classification model.

[0102] In some embodiments of this application, the process of adjusting the weight parameters and bias parameters related to the predicted target character and the predicted classification result in the trained text classification model to obtain the processed text classification model is described in detail below:

[0103] S800. Using a preset adjustment formula, the weight parameters and bias parameters related to the predicted target character and the predicted classification result in the trained text classification model are adjusted to obtain the processed text classification model.

[0104] Specifically, the adjustment formula is as follows:

[0105] H = BERT(X; θ)

[0106]

[0107]

[0108]

[0109]

[0110]

[0111] Where X represents the input of the trained text classification model, θ represents the weight parameters of the trained text classification model, BERT represents semantic encoding using the text classification model, H represents the semantic vector encoded by the text classification model, and Slice1 represents the extraction of the semantic vector encoded by the text classification model. This represents the semantic vector corresponding to the classification result. Let LN represent the semantic vector after the fully connected layer, LN represent the Layer Normalization operation, GELU represent the Gaussian error linear unit activation function, W1 represent the weight parameters of the fully connected layer, and B1 represent the bias parameters of the fully connected layer. Slice2 represents the vector matrix containing only the target characters from the dictionary, and ETable represents the vector matrix of the dictionary. This represents the score of the binary classification result corresponding to the prohibited category. B2 represents the transpose of the vector matrix containing only the target characters, and B2 represents the bias parameter for dimensionality transformation (dimensionality increase). This represents the probability of the binary classification result corresponding to the prohibited category. Softmax indicates that the probability is obtained by using the softmax function. The dictionary is a pre-established character database, and the target character is selected from the dictionary.

[0112] Where H∈R b×s×d , ETable∈R v×d ,

[0113] b represents the batch size, s represents the sequence length, d represents the vector dimension, k represents the number of prohibited categories, and v represents the dictionary size. Usually, v >> s.

[0114] As can be seen from the above technical solution, this embodiment provides an optional method of adjusting parameters. Through the above process, the parameters of the text classification model can be better adjusted to improve the prediction efficiency of the text classification model.

[0115] After adjusting the parameters of the trained text classification model using the above-described adjustment formula, the number of operations and the large number of power function calculations within the softmax function can be reduced, thus improving the prediction efficiency of the text classification model. Furthermore, verification shows that adjusting the parameters of the text classification model does not decrease the accuracy of the predicted classification results. Therefore, the text classification model obtained through the above embodiments can improve prediction efficiency while maintaining accuracy.

[0116] In some embodiments of this application, step S3, generating training data using preset hidden characters and the target text, wherein the training data is the target text in which some characters are replaced by hidden characters, is described in detail below:

[0117] S30. Replace some characters in the target text with hidden characters to obtain the replaced text.

[0118] Specifically, replacement text can be obtained in various ways. For example, hidden characters can be used to randomly replace some characters in the target text to obtain replacement text; or sensitive words in the target text can be replaced with hidden characters to obtain replacement text.

[0119] S31. If the target text does not have a label, then the replacement text is directly used as the training data.

[0120] Specifically, when there are no tags in the target text, the replacement text obtained after replacing the target text with replacement characters can be directly used as training data.

[0121] S32. If the target text has a label, then the replacement text is processed using a preset text template to obtain training data.

[0122] Specifically, if the target text contains tags, the replacement text can be processed using a text template to obtain training data, so that the prohibited category exists at a fixed position in each training data.

[0123] As can be seen from the above technical solution, this embodiment provides an optional way to generate training data. The training data generated by the above method have similar formats, which facilitates semi-supervised learning of the text classification model during the training process and speeds up the training process of the text classification model.

[0124] In some embodiments of this application, the process of step S32, which involves processing the replacement text using a preset text template to obtain training data if the target text contains tags, is described in detail below:

[0125] S320. Using the annotation tags of the target text corresponding to the replaced text, determine the binary classification result corresponding to each prohibited category.

[0126] Specifically, the annotation labels of the target text can indicate the prohibited category that the target text matches. Based on the prohibited category that the target text matches, the binary classification result corresponding to each prohibited category is determined. For example, if the annotation labels of the target text indicate that the prohibited category that the target text matches is the fifth prohibited type, and the text classification model can identify prohibited categories as the first prohibited type, the second prohibited type, the third prohibited type, the fourth prohibited type, and the fifth prohibited type, then the binary classification result corresponding to the first prohibited type can be no, the binary classification result corresponding to the second prohibited type can be no, the binary classification result corresponding to the third prohibited type can be no, the binary classification result corresponding to the fourth prohibited type can be no, and the binary classification result corresponding to the fifth prohibited type can be yes.

[0127] S321. Based on the fixed order of multiple prohibited categories in the text template, determine the order of each binary classification result.

[0128] Specifically, the order of binary classification results corresponding to each prohibited category that the text classification model can identify can be determined based on the fixed order of the prohibited categories in the text template. For example, when the prohibited categories that the text classification model can identify are the first prohibited type, the second prohibited type, the third prohibited type, the fourth prohibited type, and the fifth prohibited type, what needs to be determined is the order of the binary classification results corresponding to the first prohibited type, the second prohibited type, the third prohibited type, the fourth prohibited type, and the fifth prohibited type.

[0129] S322. Based on the order of each binary classification result, form a combination of binary classification results.

[0130] Specifically, the binary classification results can be combined according to the order in which they correspond, and the resulting combination is the binary classification result combination. For example, if the fixed order of the prohibited categories in the text template can be: first prohibited type, second prohibited type, third prohibited type, fourth prohibited type, and fifth prohibited type, then the binary classification result "no" corresponding to the first prohibited type is in the first order, the binary classification result "no" corresponding to the second prohibited type is in the second order, the binary classification result "no" corresponding to the third prohibited type is in the third order, the binary classification result corresponding to the fourth prohibited type is "no", and the binary classification result corresponding to the fifth prohibited type is "yes", resulting in a binary classification result combination of "no-no-no-no-yes".

[0131] S323. Based on the specific locations corresponding to multiple prohibited categories in the text template, the specific locations of the replacement text in the text template, the combination of the binary classification results, and the replacement text, training data is generated.

[0132] Specifically, based on the specific position of the prohibited category in the text template and the specific position of the replacement text, the binary classification result combination and the replacement text are combined to generate training data. For example, if the position of the prohibited category in the text template is located at the left end of the replacement text, then the binary classification result combination can be placed before the replacement text to generate training data.

[0133] As can be seen from the above technical solution, this embodiment provides an optional combination method for training data. Through the above method, in the process of training the text classification model, the text classification model can distinguish the replacement text and the combination of binary classification results according to the position of each component of the training data, thereby improving the training efficiency of the text classification model.

[0134] In some embodiments of this application, the process of generating training data based on step S323, the specific locations corresponding to multiple prohibited categories in the text template, the specific locations of the replacement text in the text template, the combination of the binary classification results, and the replacement text is described in detail below:

[0135] S3230. Based on the specific locations corresponding to multiple prohibited categories in the text template and the specific locations of the replacement text, combine the binary classification results and the replacement text to obtain combined data.

[0136] Specifically, the binary classification results and the replacement text can be integrated based on the prohibited categories and the position of the replacement text in the text template. The result is combined data, which contains the binary classification results and the replacement text.

[0137] S3231. Add a preset prefix character to the combined data, and add a suffix character between the binary classification result of the combined data and the replacement text to obtain training data, so that after the training data is input into the text classification model, the text classification model can distinguish and identify the binary classification result combination and the replacement text based on the prefix character and the suffix character.

[0138] Specifically, prefix and suffix characters can be pre-set. The specific content of the prefix and suffix characters can be set according to actual needs. The prefix character can contain a string indicating the activation meaning. The training data can consist of "prefix character + binary classification result combination + suffix character + replacement text". Therefore, the data between the prefix and suffix characters is the binary classification result combination, and the data after the suffix character is the replacement text.

[0139] As can be seen from the above technical solution, this embodiment provides an optional method for generating training data by combining replacement text and binary classification results. By using the above method, prefix characters and suffix characters can be used to better distinguish the combination of binary classification results and replacement text in the training data, thereby better completing the training of the text classification model.

[0140] The text classification method provided in the embodiments of this application will be described in detail below. The text classification model trained above can be applied to the text classification method provided below. The text classification method provided below can be referred to in conjunction with the text classification model training method provided above.

[0141] The specific steps of a text classification method can be summarized as follows:

[0142] S1. Obtain the text information to be classified.

[0143] Specifically, it can obtain text information that needs to be categorized.

[0144] Among these methods, text information to be categorized can be obtained from the internet, such as from chat logs in a live stream or from a chat interface.

[0145] S2. The text classification model trained by the text classification model training method provided in any of the above embodiments is used to classify the text information to be classified, and the classification result is obtained. The classification result includes binary classification results corresponding to multiple prohibited categories.

[0146] Specifically, the text information to be classified can be input into the text classification model trained above to obtain the classification result predicted by the text classification model. The classification result can contain binary classification results corresponding to multiple prohibited categories. Through the classification result, it can be determined whether the text information to be classified belongs to the prohibited text.

[0147] As can be seen from the above technical solution, this application provides a text classification method. Through the above process, illegal text and normal text can be identified so as to process illegal text and protect the physical and mental health of underage users and their access to the Internet.

[0148] The text classification model training device provided in the embodiments of this application is described below. The text classification model training device described below can be referred to in correspondence with the text classification model training method described above.

[0149] See Figure 2 , Figure 2 This is a schematic diagram of the structure of a text classification model training device disclosed in an embodiment of this application.

[0150] likeFigure 2 As shown, the text classification model training device may include:

[0151] Determining unit 1 is used to determine a training text set, which includes multiple unlabeled violation texts, multiple violation texts labeled with prohibited categories, and multiple unlabeled normal texts;

[0152] Selection unit 2 is used to sequentially select target text from the training text set;

[0153] The generation unit 3 is used to generate training data using preset hidden characters and the target text, wherein the training data is the target text in which some characters are replaced by hidden characters;

[0154] Classification unit 4 is used to input the training data into the text classification model to obtain the target character predicted by the text classification model and the classification result predicted by the text classification model based on the training data. The classification result is a binary classification result corresponding to multiple prohibited categories.

[0155] Calculation unit 5 is used to calculate the text semantic loss value of the text classification model based on the target character and the target text;

[0156] Unit 6 is used to calculate the classification loss value of the text classification model based on the classification result and the target text.

[0157] The adjustment unit 7 is used to adjust the parameters of the text classification model based on the text semantic loss value and the classification loss value until the text semantic loss value and the classification loss value meet the preset conditions, so as to obtain the trained text classification model.

[0158] Optionally, the text classification model training device may also include:

[0159] The parameter adjustment unit is used to adjust the bias parameters and weight parameters of the trained text classification model to obtain the processed text classification model. The output of the processed text classification model is the classification result corresponding to the input text.

[0160] Optionally, the parameter adjustment unit may include:

[0161] The weight parameter adjustment unit is used to adjust the weight parameters and bias parameters related to the predicted target character and the predicted classification result in the trained text classification model, so as to obtain the processed text classification model.

[0162] Optionally, the weight parameter adjustment unit may include:

[0163] The company uses a unit to adjust the weight parameters and bias parameters related to the predicted target character and the predicted classification result in the trained text classification model using a preset adjustment formula, so as to obtain the processed text classification model.

[0164] The adjustment formula is as follows:

[0165] H = BERT(X; θ)

[0166]

[0167]

[0168]

[0169]

[0170]

[0171] Where X represents the input of the trained text classification model, θ represents the weight parameters of the trained text classification model, BERT represents semantic encoding using the text classification model, H represents the semantic vector encoded by the text classification model, and Slice1 represents the extraction of the semantic vector encoded by the text classification model. This represents the semantic vector corresponding to the classification result. Let LN represent the semantic vector after the fully connected layer, LN represent the Layer Normalization operation, GELU represent the Gaussian error linear unit activation function, W1 represent the weight parameters of the fully connected layer, and B1 represent the bias parameters of the fully connected layer. Slice2 represents the vector matrix containing only the target characters from the dictionary, and ETable represents the vector matrix of the dictionary. This represents the score of the binary classification result corresponding to the prohibited category. B2 represents the transpose of the vector matrix containing only the target characters, and B2 represents the bias parameter for dimensionality transformation (dimensionality increase). This represents the probability of the binary classification result corresponding to the prohibited category, and Softmax indicates that the probability is obtained using the softmax function.

[0172] Optionally, the generating unit may include:

[0173] A character replacement unit is used to replace some characters in the target text with hidden characters to obtain the replaced text.

[0174] The label determination unit is used to directly use the replacement text as the training data if the target text does not have a label.

[0175] The text processing unit is used to process the replacement text using a preset text template if the target text has a label, thereby obtaining training data.

[0176] Optionally, the text processing unit may include:

[0177] The binary classification result determination unit is used to determine the binary classification result corresponding to each prohibited category by using the annotation label of the target text corresponding to the replacement text;

[0178] The order determination unit is used to determine the order of each binary classification result based on the fixed order of multiple prohibited categories in the text template;

[0179] The binary classification result combination unit is used to form a combination of binary classification results according to the order of the various binary classification results;

[0180] The location utilization unit is used to generate training data based on the specific locations corresponding to multiple prohibited categories in the text template, the specific locations of the replacement text in the text template, the combination of the binary classification results, and the replacement text.

[0181] Optionally, the location utilization unit may include:

[0182] The first position utilization unit is used to combine the binary classification results and the replacement text based on the specific positions corresponding to multiple prohibited categories in the text template and the specific positions of the replacement text to obtain combined data;

[0183] The second position utilization unit is used to add a preset prefix character to the combined data and add a suffix character between the binary classification result of the combined data and the replacement text to obtain training data. So that after the training data is input into the text classification model, the text classification model can distinguish and identify the binary classification result combination and the replacement text based on the prefix character and the suffix character.

[0184] The text classification model training device provided in this application embodiment can be applied to text classification model training devices, such as PC terminals, cloud platforms, servers, and server clusters. Optionally, Figure 3 The hardware structure block diagram of the text classification model training device is shown below. Figure 3 The hardware structure of a text classification model training device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4.

[0185] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0186] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0187] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0188] The memory stores a program, which the processor can call. The program is used for:

[0189] A training text set is determined, which includes multiple unlabeled violation texts, multiple violation texts labeled with prohibited categories, and multiple unlabeled normal texts;

[0190] Target texts are selected sequentially from the training text set;

[0191] Training data is generated using preset hidden characters and the target text, wherein the training data is the target text in which some characters are replaced by hidden characters;

[0192] The training data is input into a text classification model to obtain the target character predicted by the text classification model and the classification result predicted by the text classification model based on the training data. The classification result is a binary classification result corresponding to multiple prohibited categories.

[0193] Calculate the text semantic loss value of the text classification model based on the target character and the target text;

[0194] Based on the classification results and the target text, calculate the classification loss value of the text classification model;

[0195] Based on the text semantic loss value and the classification loss value, the parameters of the text classification model are adjusted until the text semantic loss value and the classification loss value meet the preset conditions, thus obtaining the trained text classification model.

[0196] Optionally, the refined and extended functions of the program can be referred to the above description.

[0197] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0198] A training text set is determined, which includes multiple unlabeled violation texts, multiple violation texts labeled with prohibited categories, and multiple unlabeled normal texts;

[0199] Target texts are selected sequentially from the training text set;

[0200] Training data is generated using preset hidden characters and the target text, wherein the training data is the target text in which some characters are replaced by hidden characters;

[0201] The training data is input into a text classification model to obtain the target character predicted by the text classification model and the classification result predicted by the text classification model based on the training data. The classification result is a binary classification result corresponding to multiple prohibited categories.

[0202] Calculate the text semantic loss value of the text classification model based on the target character and the target text;

[0203] Based on the classification results and the target text, calculate the classification loss value of the text classification model;

[0204] Based on the text semantic loss value and the classification loss value, the parameters of the text classification model are adjusted until the text semantic loss value and the classification loss value meet the preset conditions, thus obtaining the trained text classification model.

[0205] Optionally, the refined and extended functions of the program can be referred to the above description.

[0206] The text classification device provided in the embodiments of this application will be described in detail below. The text classification device described below can be referred to in conjunction with the text classification method provided above.

[0207] The text classification device may include:

[0208] The text acquisition unit is used to acquire the text information to be classified.

[0209] The information classification unit is used to classify the text information to be classified using a text classification model trained by a text classification model training method, and to obtain a classification result, wherein the classification result includes binary classification results corresponding to multiple prohibited categories.

[0210] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0211] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0212] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. The various embodiments of this application can be combined with each other. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a text classification model, characterized in that, include: A training text set is determined, which includes multiple unlabeled violation texts, multiple violation texts labeled with prohibited categories, and multiple unlabeled normal texts; Target texts are selected sequentially from the training text set; Training data is generated using preset hidden characters and the target text, wherein the training data is the target text in which some characters are replaced by hidden characters; The training data is input into a text classification model to obtain the target character predicted by the text classification model and the classification result predicted by the text classification model based on the training data. The classification result is a binary classification result corresponding to multiple prohibited categories. Calculate the text semantic loss value of the text classification model based on the target character and the target text; Based on the classification results and the target text, calculate the classification loss value of the text classification model; Based on the text semantic loss value and the classification loss value, the parameters of the text classification model are adjusted until the text semantic loss value and the classification loss value meet the preset conditions, thus obtaining the trained text classification model; The weight parameters and bias parameters related to the predicted target character and the predicted classification result in the trained text classification model are adjusted using a preset adjustment formula to obtain the processed text classification model. The output of the processed text classification model is the classification result corresponding to the input text. The adjustment formula is as follows: ； Where X represents the input of the trained text classification model. The weight parameters of the trained text classification model are represented by BERT, H represents the semantic vector encoded by the text classification model, and Slice1 represents the truncated semantic vector encoded by the text classification model. This represents the semantic vector corresponding to the classification result. Let LN represent the semantic vector after the fully connected layer, LN represent the Layer Normalization operation, GELU represent the Gaussian error linear unit activation function, W1 represent the weight parameters of the fully connected layer, and B1 represent the bias parameters of the fully connected layer. Slice2 represents the vector matrix containing only the target characters from the dictionary, and ETable represents the vector matrix of the dictionary. This represents the score of the binary classification result corresponding to the prohibited category. B2 represents the transpose of the vector matrix containing only the target characters, and B2 represents the bias parameter for dimensionality transformation (dimensionality increase). This represents the probability of the binary classification result corresponding to the prohibited category. This indicates that the probability is calculated using the softmax function, and the dictionary is a pre-built character database.

2. The text classification model training method according to claim 1, characterized in that, Training data is generated using preset hidden characters and the target text, including: The target text is replaced with hidden characters to obtain the replaced text; If the target text does not have a label, then the replacement text is directly used as the training data; If the target text has a label, the replacement text is processed using a preset text template to obtain training data.

3. The text classification model training method according to claim 2, characterized in that, The text template includes a fixed order and specific position corresponding to multiple prohibited categories, as well as the specific position of the replacement text; The replacement text is processed using a preset text template to obtain training data, including: Using the annotation tags of the target text corresponding to the replaced text, determine the binary classification result corresponding to each prohibited category; Based on the fixed order of multiple prohibited categories in the text template, the order of each binary classification result is determined; Based on the order of the various binary classification results, a combination of binary classification results is formed; Training data is generated based on the specific locations corresponding to multiple prohibited categories in the text template, the specific locations of the replacement text in the text template, the combination of the binary classification results, and the replacement text.

4. The text classification model training method according to claim 3, characterized in that, The training data is generated based on the specific locations corresponding to multiple prohibited categories in the text template, the specific locations of the replacement text in the text template, the combination of the binary classification results, and the replacement text, including: Based on the specific locations corresponding to multiple prohibited categories in the text template, as well as the specific locations of the replacement text, the binary classification results and the replacement text are combined to obtain combined data; A preset prefix character is added to the combined data, and a suffix character is added between the binary classification result of the combined data and the replacement text to obtain training data. When the training data is input into the text classification model, the text classification model distinguishes and identifies the binary classification result combination and the replacement text based on the prefix character and the suffix character.

5. A text classification model training device, characterized in that, include: The determining unit is used to determine a training text set, which includes multiple unlabeled violation texts, multiple violation texts labeled with prohibited categories, and multiple unlabeled normal texts. A selection unit is used to sequentially select target text from the training text set; The generation unit is used to generate training data using preset hidden characters and the target text, wherein the training data is the target text in which some characters are replaced by hidden characters; A classification unit is used to input the training data into a text classification model to obtain the target character predicted by the text classification model and the classification result predicted by the text classification model based on the training data. The classification result is a binary classification result corresponding to multiple prohibited categories. The calculation unit is used to calculate the text semantic loss value of the text classification model based on the target character and the target text. The unit is used to calculate the classification loss value of the text classification model based on the classification result and the target text. An adjustment unit is used to adjust the parameters of the text classification model based on the text semantic loss value and the classification loss value until the text semantic loss value and the classification loss value meet preset conditions, thereby obtaining the trained text classification model; The parameter adjustment unit is used to adjust the weight parameters and bias parameters related to the predicted target character and the predicted classification result in the trained text classification model using a preset adjustment formula, so as to obtain the processed text classification model. The output of the processed text classification model is the classification result corresponding to the input text. The adjustment formula is as follows: ； Where X represents the input of the trained text classification model. The weight parameters of the trained text classification model are represented by BERT, H represents the semantic vector encoded by the text classification model, and Slice1 represents the truncated semantic vector encoded by the text classification model. This represents the semantic vector corresponding to the classification result. Let LN represent the semantic vector after the fully connected layer, LN represent the Layer Normalization operation, GELU represent the Gaussian error linear unit activation function, W1 represent the weight parameters of the fully connected layer, and B1 represent the bias parameters of the fully connected layer. Slice2 represents the vector matrix containing only the target characters from the dictionary, and ETable represents the vector matrix of the dictionary. This represents the score of the binary classification result corresponding to the prohibited category. B2 represents the transpose of the vector matrix containing only the target characters, and B2 represents the bias parameter for dimensionality transformation (dimensionality increase). This represents the probability of the binary classification result corresponding to the prohibited category. This indicates that the softmax function is used to calculate the probability.

6. A text classification method, characterized in that, include: Obtain the text information to be classified; The text classification model trained using any one of the text classification model training methods of claims 1-4 is used to classify the text information to be classified, and the classification result is obtained, wherein the classification result includes binary classification results corresponding to multiple prohibited categories.

7. A text classification device, characterized in that, include: The text acquisition unit is used to acquire the text information to be classified. The information classification unit is used to classify the text information to be classified using the text classification model trained by any one of the text classification model training methods of claims 1-4, and to obtain a classification result, wherein the classification result includes binary classification results corresponding to multiple prohibited categories.

Images