A text classification method and device applying a LOF model of outlier detection algorithm

Abstract

Embodiments of the present application provide a text classification method and device using a LOF model. The method comprises: obtaining a training data set comprising training texts and corresponding class labels, the training texts comprising training texts with known labels and training texts with other labels configured at a preset proportion; training a classification model using the training data set; inputting the training texts with known labels into the classification model to obtain embedding representations, intermediate results and final representation vectors of the training texts with known labels; training a plurality of LOF models according to the embedding representations, intermediate results and final representation vectors of the training texts with known labels; and determining whether a test text is of an unknown class according to the classification model and the plurality of LOF models. The technical solution provided by the present application enables the text classification model to identify unknown classes from test texts, avoids assigning texts of unknown classes to known class labels, and improves the accuracy of text classification.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a text classification method and apparatus using the LOF (Outlier Detection) model. Background Technology

[0002] Text classification is one of the fundamental tasks in the field of natural language processing (NLP) technology. It has a wide range of applications in real life. For example, applications such as public opinion monitoring, news classification, and sentiment classification based on NLP technology are all achieved through text classification tasks.

[0003] Text classification tasks train a classification model using training texts of several fixed categories, enabling the model to identify these categories from unknown text. Current classification models are typically trained using deep learning models, which can only classify input text within their trained categories. For input text in untrained categories, deep learning models will assign the highest probability category among all known categories, leading to incorrect classification. For example, consider a news text classification model. If a meaningless string of characters, such as ABCDEF123456, is input, the model will classify it into a known news category and return the result. However, this meaningless string clearly does not belong to news text and therefore cannot be classified into any news category.

[0004] It is evident that current text classification tasks implemented using deep learning models cannot correctly identify text outside of known categories, and may even classify text outside of known categories into incorrect categories, leading to a decrease in classification accuracy. Summary of the Invention

[0005] This application provides a text classification method and apparatus that uses the Loop Outlier Detection (LOF) model, which can identify text of unknown categories, thereby avoiding misclassifying text of unknown categories into a known category and improving classification accuracy.

[0006] In a first aspect, embodiments of this application provide a text classification method, the method comprising: acquiring a training dataset, the training dataset including training texts and their corresponding category labels, the training texts including training texts with known labels configured in a preset ratio and training texts with other labels; training a classification model using the training dataset, the classification model using a combination of distance loss and adversarial loss between the text representation vectors of the training texts and the category labels as the total loss function for training; inputting the training texts with known labels into the classification model to obtain the embedding representations, intermediate results, and final representation vectors of the training texts with known labels; training multiple outlier detection algorithm (LOF) models based on the embedding representations, intermediate results, and final representation vectors of the training texts with known labels; and determining whether a test text belongs to an unknown category based on the classification model and the multiple LOF models.

[0007] In one implementation, multiple outlier detection algorithm LOF models are trained based on the embedding representations, intermediate results, and final representation vectors of training texts with known labels. This includes: generating multiple sample pools based on the embedding representations, intermediate results, and final representation vectors of training texts with known labels; and training an LOF model separately using each sample pool to obtain multiple LOF models with the same number of sample pools.

[0008] In one implementation, the classification model is the BERT model, and the number of sample pools is 2N+1+1, where N is the number of hidden layers in the BERT model. The 2N+1+1 sample pools specifically include: hidden state matrices of N hidden layers, attention matrices of N hidden layers, 1 embedding representation, and 1 final representation vector.

[0009] In one implementation, the classification model is an RNN model, and the number of sample pools is N+1+1, where N is the number of hidden layers in the RNN model. The N+1+1 sample pools specifically include: the final hidden state matrix of N hidden layers, 1 embedding representation, and 1 final representation vector.

[0010] In one implementation, the classification model is a CNN model, and the number of sample pools is K×N+1+1, where N is the number of hidden layers in the CNN model, K is the number of feature maps in the CNN model, and the K×N+1+1 sample pools specifically include: K×N feature map matrix results, 1 embedding representation, and 1 final representation vector.

[0011] In one implementation, determining whether a test text belongs to an unknown category based on a classification model and multiple LOF models includes: using a classification model to obtain the text representation vector of the test text; determining the test text to belong to an unknown category when the similarity between the text representation vector of the test text and the label vectors of other labels is the highest; otherwise, determining the test text to belong to an unknown category when the similarity between the text representation vector of the test text and the label vectors of all category labels is less than a preset first threshold; otherwise, using multiple LOF models to determine whether the test text belongs to an unknown category.

[0012] In one implementation, multiple LOF models are used to determine whether the test text belongs to an unknown category. This includes: inputting the embedded representation, intermediate results, and final representation vector of the test text into each LOF model to obtain the outlier factor output by each LOF model; summing the outlier factors output by each LOF model with weights to obtain a weighted outlier factor; and determining that the test text belongs to an unknown category when the weighted outlier factor is greater than a preset second threshold.

[0013] In one implementation, the text representation vector is obtained by: segmenting the text into multiple sub-texts, where the text is either training text or test text; converting the segmentation result into an encoded representation of the text; and inputting the encoded representation of the text into a classification model to obtain the text representation vector; wherein the text representation vector is the vector corresponding to the first character or word of the text, or the text representation vector is the max pooling vector of the hidden layer dimension of the sub-text, or the text representation vector is the average pooling vector of the hidden layer dimension of the sub-text.

[0014] In one implementation, the distance loss between the text representation vector of the training text and the class label is obtained by using the Enhanced Edge Cosine Loss (LMCL) function to calculate the distance loss between the text representation vector of the training text and the class label.

[0015] Secondly, embodiments of this application provide a text classification device applying the Loop Outlier Detection (LOF) algorithm. The device includes: a first acquisition module for acquiring a training dataset, which includes training texts and their corresponding category labels. The training texts include training texts with known labels and training texts with other labels configured in a preset ratio; a training module for training a classification model using the training dataset, wherein the classification model uses a combination of distance loss and adversarial loss between the text representation vectors of the training texts and the category labels as the total loss function for training; a second acquisition module for inputting the training texts with known labels into the classification model to obtain the embedding representations, intermediate results, and final representation vectors of the training texts with known labels; a post-processing module for training multiple Loop Outlier Detection (LOF) models based on the embedding representations, intermediate results, and final representation vectors of the training texts with known labels; and a classification module for determining whether a test text belongs to an unknown category based on the classification model and the multiple LOF models.

[0016] The technical solution provided in this application introduces training texts with known other labels into the dataset for training, and introduces adversarial mechanisms and the LOF method, enabling the text classification model to identify unknown categories from test texts, avoiding the assignment of texts with unknown categories to known category labels, and improving the accuracy of text classification. Attached Figure Description

[0017] Figure 1 This is a flowchart of a text classification method using the LOF model provided in an embodiment of this application;

[0018] Figure 2 This is a schematic diagram of the structure of a classification model shown in an embodiment of this application;

[0019] Figure 3 This is a flowchart of step S102 of a text classification method using the LOF model provided in an embodiment of this application;

[0020] Figure 4 This is a flowchart of step S201 of a text classification method using the LOF model provided in an embodiment of this application;

[0021] Figure 5 This is a flowchart of step S203 of a text classification method using the LOF model provided in an embodiment of this application;

[0022] Figure 6 This is a schematic diagram illustrating the intermediate results obtained by the BERT model in an embodiment of this application;

[0023] Figure 7 This is a flowchart provided in an embodiment of the present application for determining whether a test text belongs to an unknown category using the LOF model;

[0024] Figure 8 This is a schematic diagram of the structure of a text classification device using the LOF model provided in an embodiment of this application;

[0025] Figure 9 This is a schematic diagram of the structure of a text classification device using the LOF model provided in an embodiment of this application. Detailed Implementation

[0026] Text classification is one of the fundamental tasks in the field of natural language processing (NLP) technology. It has a wide range of applications in real life. For example, applications such as public opinion monitoring, news classification, and sentiment classification based on NLP technology are all achieved through text classification tasks.

[0027] Text classification tasks train a classification model using training texts from several known categories, enabling the model to classify unknown text into a known category. Current classification models are typically trained using deep learning models, which can only classify input text within a few known categories. For input text from other categories, deep learning models will assign the category with the highest probability among all known categories, leading to incorrect classification. For example, consider a news text classification model. If a meaningless string of characters, such as ABCDEF123456, is input, the model will classify it into a known news category and return a result. However, this meaningless string clearly does not belong to news text and therefore cannot be classified into any news category. For example, in a certain classification scenario, if the training text includes labels for the three categories of sports, economy, and entertainment, then the classification model trained using the training text of these three categories can only have the ability to classify these three categories. In this case, if a political text is input into the classification model, the classification model will also classify the text into one of the three categories of sports, economy, and entertainment.

[0028] It is evident that current text classification tasks implemented using deep learning models cannot correctly identify text outside of known categories, and may even classify text outside of known categories into incorrect categories, leading to a decrease in classification accuracy.

[0029] This application provides a text classification method using the LOF model, which can identify text of unknown categories, thereby avoiding misclassifying text of unknown categories into a known category and improving classification accuracy.

[0030] Figure 1 This is a flowchart of a text classification method using the LOF model provided in an embodiment of this application. Figure 1As shown, the method may include the following steps S101-S106:

[0031] Step S101: Obtain the training dataset. The training dataset includes training texts and their corresponding category labels. The training texts include training texts with known labels configured in a preset ratio and training texts with other labels.

[0032] Step S102: Train a classification model using the training dataset. The classification model uses a combination of distance loss and adversarial loss between the text representation vector of the training text and the category label as the total loss function for training.

[0033] Figure 2 This is a schematic diagram illustrating the structure of a classification model according to an embodiment of this application. For example... Figure 2 As shown, the classification model includes:

[0034] The layers consist of an embedding layer, a hidden layer, fully connected layers, a loss layer, and an adversarial layer.

[0035] The embedding layer serves as the input to the classification model, used to encode and represent the input training text (or test text, or simply text) and its corresponding label text.

[0036] Hidden layers can be common Transformers structures, including multi-head attention layers, add and normalization layers, feed forward layers, etc.

[0037] Linear layers are used to perform linear mapping classification on the output of hidden layers.

[0038] The loss function layer is used to obtain the model training loss by calculating the difference between the output of the hidden layer and the label, thereby guiding the model. In this embodiment, the loss function layer also includes a similarity calculation layer, which is used to calculate the similarity between the text representation vector of the training text and the label vector. This similarity can be, for example, cosine similarity or Euclidean distance.

[0039] Figure 3 This is a flowchart of step S102 of a text classification method using the LOF model provided in an embodiment of this application. Figure 3 As shown, in one implementation, step S102 may include the following steps S201-S204:

[0040] Step S201: Input the training text into the hidden layer of the classification model to obtain the text representation vector of the training text.

[0041] Figure 4 This is a flowchart of step S201 of a text classification method using the LOF model provided in an embodiment of this application. Figure 4 As shown, in one implementation, step S201 may include the following steps S301-S304:

[0042] Step S301: The training text is segmented into multiple sub-texts to obtain the word segmentation representation of the training text.

[0043] The segmentation of the training text can be word segmentation, character segmentation, or WordPiece segmentation, etc.

[0044] In this embodiment, the training text can be considered as an independent sentence composed of several sub-texts, which can be characters, words, phrases, etc. The vector space model represents the sub-texts as vectors or matrices composed of real-valued components, representing the semantics of the text. In recent years, researchers have begun using neural networks to extract feature representations of text. For example, assuming a classification task has I training texts, the corresponding i-th training text can be represented as x. i Training text x extracted using a neural network i The feature vector can be represented as

[0045] Step S302: Convert the word segmentation representation of the training text into an encoded representation.

[0046] In specific implementation, the codes corresponding to each sub-text in the word segmentation representation of the training text can be retrieved from the vocabulary list. The codes of each sub-text are then arranged in order to obtain the encoded representation of the training text. In this embodiment, the vocabulary list can contain a large number of characters and words in the form of a dictionary, along with the codes for each character and word. Each character and word has a unique code, and different characters and words have different codes. For example, assuming the vocabulary list contains ten thousand characters and words, the code for the first character or word could be 0001, the code for the second character or word could be 0002, and so on. Thus, in step S302, the codes of each sub-text can be retrieved from the vocabulary list by looking up a table, thereby obtaining the encoded representation of the training text.

[0047] Step S303: Input the encoded representation of the training text into the language model to obtain the hidden layer output of the language model.

[0048] In practice, the input vector of the training text can be fed into a pre-trained language model based on Transformers, such as BERT or RoBERTa. When using a Transformers-based model, the input to this language model also includes the text category vector corresponding to the training text and a vector representing the sentence length.

[0049] Step S304: Obtain the text representation vector of the training text based on the output of the hidden layer of the language model.

[0050] The embodiments of this application can obtain the training text x in various ways based on the hidden layer output of the language model. i Text representation vector The following four implementation methods are provided as examples:

[0051] The first implementation method uses text representation vectors. For training text x i The vector corresponding to the first character or word. In this implementation, the text representation vector... Specifically, it can be generated using pre-trained language models based on Transformers, such as BERT and RoBERTa.

[0052] In practice, we can first process the training text x i The text is segmented (either by word segmentation, character segmentation, or WordPiece segmentation), then the segmented subtexts are encoded and input into a language model. Finally, the training text x is extracted from the language model's output. i The vector corresponding to the first character or word is used as the text representation vector. If the model is a Transformers model such as BERT, then the vector at the "[CLS]" position is directly obtained as the text representation vector. Example: Training text x i =“Olympic Men's Relay”, the corresponding subtext segmented using WordPiece is “[CLS]|Olympic|Meeting|Men's|Relay|[SEP]”, where “|” represents the separator between subtexts, and “[CLS]” and “[SEP]” are special characters built into the BERT vocabulary, representing the beginning of text and the separator between two sentences, respectively. BERT uses the vector at the “[CLS]” position for classification, which can be understood as the vector at this position representing sentence information; that is, the vector at the “[CLS]” position serves as the text representation vector.

[0053] The second implementation method is to use text representation vectors. This is the max-pooling vector for the hidden layer dimension of the subtext.

[0054] In the specific implementation, the training text x can be... i After sentence segmentation, the hidden layer output vector corresponding to each sub-text is obtained through the language model, and max pooling is performed on each vector along the hidden layer dimension to obtain the text representation vector. For example, for the text segmentation "Olympics|Men's|Relay", the hidden layer output vectors obtained by the model are respectively Then the text represents a vector. That is, the maximum value of all sub-text vector elements in the hidden layer output dimension.

[0055] The third implementation method is text representation vector. This is the average pooling vector for the hidden layer dimension of the subtext.

[0056] In the specific implementation, the training text x can be... i After sentence segmentation, the hidden layer output vector corresponding to each sub-text is obtained through the language model, and the average pooling of each vector in the hidden layer dimension is taken as the text representation vector. For example, for the text segmentation "Olympics|Men's|Relay", the hidden layer vectors obtained by the model are respectively Then the text represents a vector. That is, the mean of all sub-text vector elements in the hidden layer output dimension.

[0057] Step S202: The distance loss between the text representation vector of the training text and the category label is calculated using the enhanced edge cosine loss function (LMCL).

[0058] This application embodiment uses a vectorized representation of labels (referred to as Label Embedding). Each category label can be represented as a vector (or matrix), i.e., a label vector (or matrix). The dimension of the label vector (matrix) is the same as the dimension of the text representation vector, i.e., the number of hidden layer units in the classification model. In this way, the text representation vector and the label vector reside in the same vector space, and the label vector is also a semantic representation of the label. Therefore, this application embodiment can transform the traditional text classification task of "mapping text representation vectors to label probabilities" into a matching task between text representation vectors (matrices) and label (matrix) vectors.

[0059] In this embodiment, each label can correspond to one or more label vectors or matrices. For example, suppose a classification task has N labels, and the corresponding label vectors are l1, l2, ..., l N .

[0060] For the tag matrix Le, the embodiments of this application provide the following five exemplary generation methods:

[0061] The first generation method involves generating label vectors for each category. i The label matrix Le is generated using random initialization.

[0062] The second generation method involves adding a descriptive text to each category's label, and then using a language model (which can be the same method described above for generating text representation vectors) to generate the descriptive text for each label. (Any implementation method) generates descriptive text vectors, and the descriptive text vectors of each tag are used to form a tag matrix Le. For example, for the tag "sports", its descriptive text could be "sports news".

[0063] The third generation method involves adding multiple descriptive texts to each category's label, and then using a language model (which can be the same method described above for generating text representation vectors) to generate the descriptive texts for each label. (Any implementation method) generates descriptive text vectors, and the descriptive text vectors of each tag are used to form a tag matrix Le. It can be understood that since multiple descriptive texts are added to each tag, the number of dimensions corresponding to each tag in the tag matrix Le increases, specifically the number of descriptive texts. For example, for the tag "sports news", its dimensions can be enriched to ["sports news", "Olympic news", "football news"], etc. When calculating the similarity between tag representations and text representations using this method, for all descriptive texts of a single tag, the similarity representation of the descriptive text with the highest similarity is taken as the similarity for that tag.

[0064] The fourth generation method, building upon the second method, replaces the descriptive text with richer content. For example, the descriptive text for the tag "sports news" can be enriched to "Sports news reports on recent events in sports. News about sports competitions takes the lead." This way, the descriptive text contains more information related to "sports news," enriching the semantic representation of each tag in the tag matrix Le.

[0065] The fifth generation method involves identifying at least one high-frequency word for each category's label, and then using a language model (which can employ the methods described above for generating text representation vectors) to generate the high-frequency words for each label. (Any implementation method) generates high-frequency word vectors, and constructs a label matrix Le from the high-frequency word vectors of each label. This method can be seen as replacing the descriptive text with high-frequency words based on the second generation method. For example, the descriptive text of the label "sports news" can be replaced with the following high-frequency words: swimming, Olympics, basketball, football, etc.

[0066] In the specific implementation, the training text x iText representation vector The distance loss between the label and the category label is calculated using the following formula:

[0067]

[0068] in,

[0069]

[0070] Where: L lmc The distance loss is given by N, where N is the number of training texts and x is the distance loss. i For the i-th training text, y i For training text x i Authentic labels, l i For label y i The label vector, l i T For l i transpose, For training text x i The text representation vector, cos(y i ,x i ) represents the training text x i Rather than the real label y i Cosine similarity between them, y j For training text x i Real Labels i Other tags besides l j For other tags y j The label vector, l j T For l j The transpose of cosy j ,x i (x) is the training text i Other tags y j The cosine similarity between the training texts is calculated, where m is a predefined boundary parameter, s is a predefined multiple, and ||*|| denotes the L2 norm of the vector or matrix. This loss function requires training text x to... i Rather than the real label y i The similarity between them is less than m, and they are similar to other tags y. j The similarity should be minimized.

[0071] It should be noted that if there are multiple label description texts for each category, you can obtain the label description text with the highest similarity for each category, and then calculate the loss distance.

[0072] Step S203: Obtain the adversarial loss between the text representation vector of the training text and the category label.

[0073] Figure 5 This is a flowchart of step S203 of a text classification method using the LOF model provided in an embodiment of this application. Figure 5 As shown, in one implementation, step S203 may include the following steps S401-S402:

[0074] Step S401: Perform a linear mapping on the text representation vector of the training text to obtain the adversarial vector of the training text.

[0075] In the specific implementation, the training text x can be... i Text representation vector A linear layer is used to map to a set of numerical labels, where each value represents the probability that it belongs to the corresponding label, thus obtaining the training text x. i The probability of belonging to each category is mapped, and the resulting vector is used as the adversarial vector. Right now:

[0076]

[0077] in, For training text x i adversarial vector, For training text x i The text representation vector, as can be understood, is used because the text representation vector... The label values ​​were obtained using linear layer mapping; therefore, the adversarial vector... It is a vector whose length is equal to the number of labels.

[0078] Step S402: Use the cross-entropy loss function to obtain the adversarial loss between the adversarial vector of the training text and the class label.

[0079] In the specific implementation, the training text x i adversarial vector With category label y i The losses from the confrontation can be calculated using the following formula:

[0080]

[0081] Among them, L Adv To combat the losses, For training text x i The adversarial vector, y i To train the category labels corresponding to the text, adversarial vector Expectations Represents adversarial vector The expected value is output through the discriminator network, where N is the number of labels, and max...θ This indicates that the discriminator output is optimized towards the maximum value. This indicates that the second loss function seeks optimization in the direction of minimization, where λ is a known hyperparameter that can be set manually.

[0082] Step S204: Combine the distance loss and adversarial loss as the total loss function for training.

[0083] In a practical implementation, the total loss function can be:

[0084] L = L lmc +αL Adv

[0085] Among them, L lmc For distance loss, L Adv To counteract the loss, L is the total loss function, and α is a variable hyperparameter. The value of α can be 5e-3 or other values, which are not limited in this embodiment.

[0086] Step S103: Input the training text with known labels into the classification model to obtain the embedding representation, intermediate results and final representation vector of the training text with known labels.

[0087] Step S104: Generate multiple sample pools based on the embedding representations, intermediate results, and final representation vectors of the training texts with known labels.

[0088] In this embodiment, each type of data in the embedded representation, intermediate results, and final representation vector of the training text can be generated into a separate sample pool. Therefore, multiple sample pools can be obtained. The number of sample pools is related to the type and structure of the classification model. The classification model can be a BERT model, a recurrent neural network (RNN) model, or a convolutional neural network (CNN) model, etc. The following examples illustrate this in detail.

[0089] Figure 6 This is a schematic diagram illustrating the intermediate results obtained by the BERT model in an embodiment of this application. Figure 6As shown, when the classification model is BERT, the number of sample pools is 2N+1+1, where N is the number of hidden layers in the BERT model. Specifically, the embedding representations of the training text with known labels constitute one sample pool, the final representation vectors of the training text with known labels constitute another sample pool, the hidden state matrices of each of the N hidden layers constitute one sample pool, and the attention matrices of each of the N hidden layers constitute one sample pool. Therefore, the 2N+1+1 sample pools specifically include: the hidden state matrices of N hidden layers, the attention matrices of N hidden layers, 1 embedding representation, and 1 final representation vector. For example, when N=12, a total of 12×2+1+1=26 sample pools can be formed.

[0090] When the classification model is an RNN model, the number of sample pools is N+1+1, where N is the number of hidden layers in the RNN model. Specifically, the embedding representations of the training text with known labels constitute one sample pool, the final representation vectors of the training text with known labels constitute another sample pool, and the Last Hidden State matrix of each of the N hidden layers constitutes a separate sample pool. Therefore, the N+1+1 sample pools specifically include: the final hidden state matrices of the N hidden layers, 1 embedding representation, and 1 final representation vector. For example, when N=3, a total of 3+1+1=5 sample pools can be formed.

[0091] When the classification model is a CNN model, the number of sample pools is K×N+1+1, where N is the number of hidden layers in the CNN model and K is the number of feature maps in the CNN model. Specifically, the embedding representations of the training text with known labels constitute one sample pool, the final representation vectors of the training text with known labels constitute another sample pool, and the feature map matrix results from each of the N hidden layers each constitute a separate sample pool. Therefore, the K×N+1+1 sample pools specifically include: K×N feature map matrix results, 1 embedding representation, and 1 final representation vector. For example, when N=5 and K=3, a total of 15+1+1=17 sample pools can be formed.

[0092] Step S105: Train an outlier detection algorithm LOF model separately for each sample pool to obtain multiple LOF models with the same number of sample pools.

[0093] The LOF (Location-Oriented Forecasting) method is a typical density-based high-precision outlier detection method. In the LOF method, each data point is assigned an outlier factor (LOF) that depends on the neighborhood density, thus determining whether the data point is an outlier. If the LOF is much greater than 1, the data point is an outlier; if the LOF is close to 1, the data point is a normal data point.

[0094] In this embodiment, each sample pool is used to train a single LOF model, thus allowing for a total of multiple LOF models equal to the number of sample pools. For example, when the number of sample pools is 26, a total of 26 LOF models can be obtained.

[0095] Step S106: Determine whether the test text belongs to an unknown category based on the classification model and the LOF model.

[0096] For test text x test You can first input it into a classification model to obtain the test text x. test Text representation vector Then, one or a combination of the following three judgment methods can be used to judge the test text x. test Is the text of an unknown category?

[0097] The first method of judgment is when the test text x test Text representation vector The test text x is determined when the similarity between its tag vector and those of other tags is maximized. test Category unknown.

[0098] In the specific implementation, the test text x can be calculated. test Text representation vector The similarity value between the test text x and the label vectors of each category label is used to determine the similarity of the test text x. test Text representation vector Does it have the highest similarity to the tag vectors of other tags? If so, then determine the test text x. test The category is unknown. The test text x is... test Text representation vector The similarity between label vectors and other labels can be measured using cosine similarity or Euclidean distance. When using cosine similarity, the highest similarity is achieved when the test text x... test Text representation vector The test text x has the highest cosine similarity value with the label vectors of other labels. When using Euclidean distance, the highest similarity refers to the test text x. test Text representation vector It has the smallest Euclidean distance between its label vector and those of other labels.

[0099] The second method of judgment is when the test text x test Text representation vector When the similarity between the test text x and the label vectors of all category labels is less than a preset first threshold, the test text x is determined. test Unknown category.

[0100] The second judgment method can be implemented alone or after the first judgment method. For example, when the first judgment method is used to determine the test text x... test Text representation vector If the similarity between the tag vectors of other tags is not the maximum, a second judgment method can be performed. Specifically, when using cosine similarity, a similarity less than a preset first threshold means the cosine similarity is less than the preset threshold for that tag; when using Euclidean distance, a similarity less than the preset first threshold means the Euclidean distance is greater than the preset threshold for that tag.

[0101] The third method uses the LOF model to judge the test text x. test Whether it belongs to an unknown category can be determined independently or after the first and / or second determination methods. For example: if the first and / or second determination methods fail to determine the test text x... test When determining whether a category is unknown, a third method of judgment can be used.

[0102] Figure 7 This is a flowchart provided in an embodiment of this application for determining whether a test text belongs to an unknown category using the LOF model. For example... Figure 7 As shown, the process may include the following steps S501-S503:

[0103] Step S501, test text x test The embedded representation, intermediate results, and final representation vector are input into each LOF model to obtain the outlier factor output by each LOF model.

[0104] For example, when 26 LOF models are trained through step S105, the test text x test The embedded representation, intermediate results, and final representation vector can be input into 26 LOF models to obtain 26 outlier factors.

[0105] Step S502: The outlier factors output by each LOF model are weighted and summed to obtain the weighted outlier factor.

[0106] In practice, the weights of each LOF model can be determined based on the number of layers in the LOF model. The more layers, the greater the weights, and the fewer layers, the smaller the weights.

[0107] Step S503: When the weighted outlier factor is greater than a preset second threshold, determine the test text x. test Unknown category.

[0108] Additionally, if the test text x is still not determined after trying the above three methods... testIf the category is unknown, the test text x can be classified using the conventional classification method. test It is classified into the known label that has the highest similarity to it.

[0109] In this embodiment, the three judgment methods described above can be used in combination, and they are used sequentially: first the first judgment method, then the second, and finally the third. This way, if the test text x has already been determined using the first two judgment methods... test If the category is unknown, there is no need to use the third judgment method, which can reduce the problem of slow LOF model integration speed and improve classification efficiency.

[0110] The text classification method provided in this application introduces training texts with known labels into the dataset for training, and introduces adversarial mechanisms and the LOF method, enabling the text classification model to identify unknown categories from test texts, avoiding the assignment of texts with unknown categories to known category labels, and improving the accuracy of text classification.

[0111] The above embodiments describe various schemes of the text classification method provided in this application. It is understood that each device or module includes corresponding hardware structures and / or software modules to perform the above functions. Those skilled in the art should readily recognize that, in conjunction with the hardware and method steps described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0112] Figure 8 This is a schematic diagram of the structure of a text classification device applying the LOF model, as provided in an embodiment of this application. Figure 8 As shown, the device includes a hardware module for implementing the text classification method provided in the embodiments of this application, including: a processor 610 and a memory 620. The memory 620 includes program instructions 630. When the program instructions 630 are executed by the processor 610, the text classification device performs the following method steps:

[0113] Obtain the training dataset, which includes training texts and their corresponding category labels. The training texts include training texts with known labels configured in a preset ratio and training texts with other labels.

[0114] The classification model is trained using the training dataset. The classification model uses a combination of distance loss and adversarial loss between the text representation vectors of the training text and the class labels as the total loss function for training.

[0115] The training text with known labels is input into the classification model to obtain the embedding representation, intermediate results and final representation vector of the training text with known labels;

[0116] Multiple outlier detection algorithm LOF models are trained based on the embedding representations, intermediate results, and final representation vectors of training texts with known labels.

[0117] The test text is determined to be of an unknown category based on the classification model and multiple LOF models.

[0118] Figure 9 This is a schematic diagram of the structure of a text classification device applying the LOF model, as provided in an embodiment of this application. Figure 9 As shown, the device includes a software module for implementing the text classification method provided in the embodiments of this application, including:

[0119] The first acquisition module 710 is used to acquire a training dataset. The training dataset includes training texts and their corresponding category labels. The training texts include training texts with known labels configured in a preset ratio and training texts with other labels.

[0120] Training module 720 is used to train a classification model using the training dataset. The classification model uses a combination of distance loss and adversarial loss between the text representation vectors of the training text and the class labels as the total loss function for training.

[0121] The second acquisition module 730 is used to input training text with known labels into the classification model to obtain the embedding representation, intermediate results and final representation vector of the training text with known labels;

[0122] Post-processing module 740 is used to train multiple outlier detection algorithm LOF models based on the embedding representations, intermediate results and final representation vectors of training text with known labels;

[0123] The classification module 750 is used to determine whether the test text belongs to an unknown category based on the classification model and multiple LOF models.

[0124] The text classification device provided in this application introduces training texts with known labels into the dataset for training, and introduces adversarial mechanisms and the LOF method, enabling the text classification model to identify unknown categories from test texts, avoiding the assignment of texts with unknown categories to known category labels, and improving the accuracy of text classification.

[0125] It is readily understood that, based on the several embodiments provided in this application, those skilled in the art can combine, split, or reorganize the embodiments of this application to obtain other embodiments, none of which exceed the protection scope of this application.

[0126] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A text classification method using the Loop Outlier Detection (LOF) model, characterized in that, include: Obtain a training dataset, which includes training texts and their corresponding category labels. The training texts include training texts with known labels configured in a preset ratio and training texts with other labels. A classification model is trained using a training dataset, wherein the classification model uses a combination of distance loss and adversarial loss between the text representation vector of the training text and the category label as the total loss function for training. Training text with known labels is input into the classification model to obtain the embedding representation, intermediate results, and final representation vector of the training text with known labels; Multiple LOF models are trained based on the embedding representations, intermediate results, and final representation vectors of the training text with known labels; The step of training multiple outlier detection algorithm LOF models based on the embedding representations, intermediate results, and final representation vectors of the training text with known labels includes: Multiple sample pools are generated based on the embedding representation, intermediate results, and final representation vector of the training text with known labels; Each of the sample pools is used to train a separate LOF model to obtain multiple LOF models with the same number of sample pools. The test text is determined to be of an unknown category based on the classification model and multiple LOF models.

2. The method according to claim 1, characterized in that, The classification model is the BERT model, and the number of sample pools is 2N+1+1, where N is the number of hidden layers in the BERT model. The 2N+1+1 sample pools specifically include: hidden state matrices of N hidden layers, attention matrices of N hidden layers, 1 embedding representation, and 1 final representation vector.

3. The method according to claim 1, characterized in that, The classification model is an RNN model, and the number of sample pools is N+1+1, where N is the number of hidden layers in the RNN model. The N+1+1 sample pools specifically include: the final hidden state matrix of N hidden layers, 1 embedding representation, and 1 final representation vector.

4. The method according to claim 1, characterized in that, The classification model is a CNN model, and the number of sample pools is K×N+1+1, where N is the number of hidden layers in the CNN model, K is the number of feature maps in the CNN model, and the K×N+1+1 sample pools specifically include: K×N feature map matrix results, 1 embedding representation, and 1 final representation vector.

5. The method according to claim 1, characterized in that, The step of determining whether the test text belongs to an unknown category based on the classification model and multiple LOF models includes: Use the classification model to obtain the text representation vector of the test text; When the similarity between the text representation vector of the test text and the label vectors of the other labels is the greatest, the test text is determined to be of an unknown category; Otherwise, when the similarity between the text representation vector of the test text and the label vectors of all the category labels is less than a preset first threshold, the test text is determined to be of an unknown category; Otherwise, multiple LOF models are used to determine whether the test text belongs to an unknown category.

6. The method according to claim 5, characterized in that, The step of using multiple LOF models to determine whether the test text belongs to an unknown category includes: The embedding representation, intermediate results, and final representation vector of the test text are input into each of the LOF models to obtain the outlier factor output by each of the LOF models. The weighted outlier factor is obtained by summing the outlier factors output by each of the LOF models using weighted summation. When the weighted outlier factor is greater than a preset second threshold, the test text is determined to be of an unknown category.

7. The method according to claim 1, characterized in that, The text representation vector is obtained through the following method: The text is divided into multiple sub-texts, where the text is either the training text or the test text; Convert the text segmentation results into an encoded representation of the text; The encoded representation of the text is input into the classification model to obtain the text representation vector; Wherein, the text representation vector is the vector corresponding to the first character or word of the text, or the text representation vector is the max pooling vector of the hidden layer dimension of the subtext, or the text representation vector is the average pooling vector of the hidden layer dimension of the subtext.

8. The method according to claim 1, characterized in that, The distance loss between the text representation vector of the training text and the category label is obtained by using the Enhanced Edge Cosine Loss (LMCL) function to calculate the distance loss between the text representation vector of the training text and the category label.

9. A text classification device using the Loop Outlier Detection (LOF) model, characterized in that, include: The first acquisition module is used to acquire a training dataset, which includes training texts and their corresponding category labels. The training texts include training texts with known labels configured in a preset ratio and training texts with other labels. The training module is used to train a classification model using a training dataset, wherein the classification model uses a combination of distance loss and adversarial loss between the text representation vector of the training text and the category label as the total loss function for training; The second acquisition module is used to input training text with known labels into the classification model to obtain the embedding representation, intermediate results and final representation vector of the training text with known labels; The post-processing module is used to train multiple LOF models based on the embedding representations, intermediate results, and final representation vectors of the training text with known labels; The post-processing module is specifically used for: Multiple sample pools are generated based on the embedding representation, intermediate results, and final representation vector of the training text with known labels; Each of the sample pools is used to train a separate LOF model to obtain multiple LOF models with the same number of sample pools. The classification module is used to determine whether the test text belongs to an unknown category based on the classification model and multiple LOF models.

Images