A text single-class classification method, system, computer device and storage medium

Abstract

The application provides a text single-class classification method, system, computer device and storage medium. The method comprises the following steps: obtaining a part-of-speech annotated text set by performing part-of-speech annotation on a single-class text set; replacing strong semantic words obtained based on a plurality of part-of-speech self-encoders constructed according to the part-of-speech annotated text set with preset non-semantic words to obtain a weak semantic difficult sample set; training a domain feature extractor by performing class-level supervised contrast learning training on a pre-training model according to the single-class text set and the weak semantic difficult sample set; constructing a domain feature prototype according to the domain feature extractor and extracting a to-be-identified domain feature of a to-be-identified text; performing similarity calculation on the to-be-identified domain feature and the domain feature prototype to obtain a to-be-identified text domain score; and obtaining a to-be-identified text classification result in combination with a preset score threshold. The weak semantic difficult negative sample generated based on part-of-speech analysis can greatly improve the contrast learning training effect, thereby effectively improving the classification accuracy of the text single class.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, system, computer device, and storage medium for single-class text classification based on weak semantic samples. Background Technology

[0002] In today's big data era, the amount of information represented by text is also growing rapidly, and textual information contains certain valuable information. How to extract useful thematic information from a large amount of text has become a research project that many scholars are keen to study.

[0003] Existing methods for single-class text classification mainly include the N+1 classification method, partition-based methods, discriminant methods based on reconstruction error, and adversarial generative methods. Among these, the N+1 classification method optimizes the original classifier by grouping all out-of-domain samples into a new class, thus adding a new category to its predictions. However, its poor convergence is due to the diversity of out-of-domain samples, which may prevent them from clustering into a single class. Partition-based methods separate in-domain and out-of-domain samples by finding one or more optimal partitioning hyperplanes or hyperspheres in the feature space. They use support vectors far from the center point to construct class descriptions, which can easily lead to overfitting or underfitting when the sample size is small. The problem is that the discrimination method based on reconstruction error is a single-class classification method that uses an autoencoder as its core and selects the common significant features of positive samples during feature dimensionality reduction and recovery. The adversarial generative method, which uses an adversarial generative network as its core and continuously generates negative samples during adversarial training to iteratively improve the generator and discriminator, both input the whole text into a pre-trained model or recurrent neural network to obtain text features. They fail to make good use of the different characteristics of fine-grained words and domain relevance to obtain semantic representations that are fully suitable for single-class classification tasks and fine-tune the pre-trained model. They cannot give full play to their text understanding ability and ensure the accuracy of single-class classification of text. Summary of the Invention

[0004] The purpose of this invention is to provide a single-class text classification method. This method utilizes a contrastive learning approach based on weakly semantically negative samples to deeply explore the association between fine-grained words of various parts of speech and the domain to which the text belongs. Data augmentation is applied to the original samples to generate difficult negative samples that are highly similar to the original samples but have low domain relevance. Then, contrastive learning is used to fine-tune the pre-trained model, resulting in a better domain feature extractor. This allows for the direct calculation of the similarity between the domain features of the text to be identified and the domain prototype features, leading to the corresponding text classification result. This addresses the application shortcomings of single-class text classification methods, and by leveraging the weakly semantically negative samples generated from part-of-speech analysis, it significantly improves the training effect of contrastive learning, thereby effectively enhancing the classification accuracy of single-class text.

[0005] To achieve the above objectives, it is necessary to provide a text single-category classification method, system, computer device, and storage medium to address the aforementioned technical problems.

[0006] In a first aspect, embodiments of the present invention provide a text single-category classification method, the method comprising the following steps:

[0007] Obtain a single-classification text set and perform part-of-speech tagging on the single-classification text set to obtain a part-of-speech tagged text set;

[0008] Based on the part-of-speech-tagged text set, multiple part-of-speech autoencoders are constructed, and based on each part-of-speech autoencoder, strong semantic words of the part-of-speech in the single-classification text set are obtained;

[0009] Replace all strongly semantic words in the single-classification text set with preset words of equal length that have no semantic meaning to obtain the corresponding weak semantic difficult sample set.

[0010] Based on the single-class text set and the weak semantic hard sample set, a pre-trained model is trained by category-level supervised contrastive learning to obtain a domain feature extractor, and based on the domain feature extractor, a domain feature prototype corresponding to the single-class text set is constructed.

[0011] The text to be identified is input into the domain feature extractor for feature extraction to obtain the corresponding domain features to be identified. The similarity between the domain features to be identified and the domain feature prototype is calculated to obtain the corresponding domain score of the text to be identified.

[0012] Based on the domain score of the text to be identified and the preset scoring threshold, the corresponding classification result of the text to be identified is obtained.

[0013] Further, the step of performing part-of-speech tagging on the single-classification text set to obtain a part-of-speech-tagged text set includes:

[0014] Based on the preset target parts of speech and the FastHan tool, the training text samples in the single-classification text set are labeled with parts of speech to obtain the part-of-speech labeled text set; the preset target parts of speech include nouns, verbs and proper nouns.

[0015] Furthermore, the part-of-speech autoencoder includes a noun autoencoder, a verb autoencoder, and a proper noun autoencoder;

[0016] The step of constructing multiple part-of-speech autoencoders based on the part-of-speech tagged text set includes:

[0017] The part-of-speech-tagged text set is input into the pre-trained model for word meaning extraction to obtain the corresponding word semantic feature set;

[0018] Based on the part-of-speech category, the semantic feature set of the words is classified to obtain multiple part-of-speech semantic feature sets;

[0019] The pre-defined autoencoder is trained based on each part-of-speech semantic feature set to obtain the corresponding part-of-speech autoencoder.

[0020] Furthermore, the preset self-encoder includes four fully connected layers;

[0021] The step of training the preset autoencoder based on each part-of-speech semantic feature set to obtain the corresponding part-of-speech autoencoder includes:

[0022] Based on the part-of-speech semantic feature set, the preset autoencoder is trained using the reconstruction mean squared error loss function to obtain the corresponding part-of-speech autoencoder; the reconstruction mean squared error loss function is expressed as:

[0023] L = argmin||X - dec[enc(X)]|| 2

[0024] Where L represents the mean squared error loss of reconstruction; X represents the part-of-speech semantic features in the part-of-speech semantic feature set; enc(X) and dec(enc(X)) represent the encoded features and the corresponding decoded reconstructed features obtained by the part-of-speech semantic features X through a preset autoencoder, respectively.

[0025] Furthermore, the step of obtaining strongly semantic words with part-of-speech tags from the single-classification text set based on each part-of-speech autoencoder includes:

[0026] Each word semantic feature in each part-of-speech semantic feature set is input into the corresponding part-of-speech autoencoder for feature reconstruction, and the corresponding word reconstruction mean square error value is obtained.

[0027] The mean squared error values ​​of the reconstruction of each word in the same part-of-speech semantic feature set are arranged in ascending order, and words of a predetermined proportion are selected from top to bottom as the corresponding strong part-of-speech semantic words.

[0028] Further, the step of performing category-level supervised contrastive learning training on the preset pre-trained model based on the single-classification text set and the weak semantic hard sample set to obtain the domain feature extractor includes:

[0029] The pre-trained model is initialized based on the network structure and parameters of the chinese-roberta-wwm-ext model;

[0030] The single-classification text set and the weak semantic difficulty sample set are respectively used as the positive class sample set and the negative class sample set;

[0031] Each positive sample in the positive sample set is used as an anchor sample, and other positive samples in the same batch are used as positive samples. Then, weak semantic negative samples with the same number as the positive samples are selected in the negative sample set. The pre-trained model is then trained by backpropagation according to a preset loss function to obtain the domain feature extractor.

[0032] Further, the step of constructing the domain feature prototype corresponding to the single-classification text set based on the domain feature extractor includes:

[0033] Each positive class sample in the single-class text set is input into the domain feature extractor for feature extraction to obtain the corresponding domain features;

[0034] Calculate the average value of all domain features to obtain the domain feature prototype.

[0035] Secondly, embodiments of the present invention provide a text single-category classification system, the system comprising:

[0036] The preprocessing module is used to obtain a single-classification text set and perform part-of-speech tagging on the single-classification text set to obtain a part-of-speech tagged text set;

[0037] The strong semantic word acquisition module is used to construct multiple part-of-speech autoencoders based on the part-of-speech labeled text set, and to acquire the strong semantic words of the part-of-speech in the single-classification text set based on each part-of-speech autoencoder;

[0038] The negative sample generation module is used to replace all strongly semantic words in the single-classification text set with preset words of equal length without semantic meaning to obtain the corresponding weak semantic difficult sample set.

[0039] The contrastive learning training module is used to perform category-level supervised contrastive learning training on a preset pre-trained model based on the single-classification text set and the weak semantic difficulty sample set, to obtain a domain feature extractor, and to construct a domain feature prototype corresponding to the single-classification text set based on the domain feature extractor.

[0040] The domain scoring module is used to input the text to be identified into the domain feature extractor for feature extraction, obtain the corresponding domain features to be identified, and calculate the similarity between the domain features to be identified and the domain feature prototype to obtain the corresponding domain score of the text to be identified.

[0041] The result generation module is used to obtain the corresponding classification result of the text to be identified based on the domain score of the text to be identified and the preset score threshold.

[0042] Thirdly, embodiments of the present invention also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0043] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0044] This application provides a method, system, computer device, and storage medium for single-class text classification. The method involves: performing part-of-speech tagging on an acquired single-class text set to obtain a tagged text set; constructing multiple part-of-speech autoencoders based on the tagged text set; obtaining strong semantic words from the single-class text set based on each autoencoder; replacing all strong semantic words in the single-class text set with pre-defined semantically neutral words of equal length to obtain a corresponding weak semantically difficult sample set; training a pre-trained model using class-level supervised contrastive learning based on the single-class text set and the weak semantically difficult sample set to obtain a domain feature extractor; constructing a domain feature prototype corresponding to the single-class text set based on the domain feature extractor; and after inputting the text to be identified into the domain feature extractor to extract the corresponding domain features to be identified, calculating the similarity between the domain features to be identified and the domain feature prototype to obtain the corresponding domain score for the text to be identified. Finally, based on the domain score and a pre-defined scoring threshold, the corresponding classification result for the text to be identified is obtained. Compared with existing technologies, this single-class text classification method, through contrastive learning based on weak semantic negative samples, deeply explores the association between fine-grained words of various parts of speech and the domain to which the text belongs. It performs data augmentation on the original samples to generate difficult negative samples with high similarity to the original samples but low domain relevance. Then, it uses contrastive learning to fine-tune the pre-trained model to obtain a better domain feature extractor. This achieves the goal of being able to identify samples outside the domain by directly calculating the similarity between the domain features of the text to be identified and the domain prototype features. This can greatly improve the training effect of contrastive learning and thus effectively improve the classification accuracy of single-class text. Attached Figure Description

[0045] Figure 1 This is a schematic diagram illustrating the application scenario of the text single-category classification method in this embodiment of the invention;

[0046] Figure 2 This is a schematic diagram of the framework for single-category text classification in an embodiment of the present invention;

[0047] Figure 3 This is a flowchart illustrating the text single-category classification method in an embodiment of the present invention;

[0048] Figure 4This is a schematic diagram of the semantic representation of words in an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram illustrating the generation of weak semantically difficult samples in an embodiment of the present invention;

[0050] Figure 6 This is a schematic diagram illustrating class-level supervised contrastive learning of a pre-trained model in an embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram illustrating the verification results of the method of the present invention and existing methods in terms of optimal recognition performance;

[0052] Figure 8 This is a schematic diagram illustrating the verification results of the method of the present invention and existing methods in terms of recognition performance with different numbers of training samples;

[0053] Figure 9 This is a schematic diagram illustrating the verification results of the method of the present invention and existing methods in a scenario where the training samples are contaminated.

[0054] Figure 10 This is a schematic diagram of the structure of a single-category text classification system in an embodiment of the present invention;

[0055] Figure 11 This is an internal structural diagram of the computer device in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and beneficial effects of this application clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the embodiments described below are only part of the embodiments of the present invention and are used to illustrate the present invention, but are not intended to limit the scope of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0057] The text single-category classification method provided by this invention can be applied to, for example... Figure 1 The terminal or server shown. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The server can be implemented using the features provided by this invention, depending on the actual application requirements. Figure 2 The text single-category classification architecture shown herein enables efficient and accurate text single-category classification research, and the obtained text classification results are used for subsequent research on the server or transmitted to the terminal for end users to view and analyze; the following embodiments will provide a detailed description of the text single-category classification method of the present invention.

[0058] In one embodiment, such as Figure 3 As shown, a text single-category classification method is provided, including the following steps:

[0059] S11. Obtain a single-classification text set and perform part-of-speech tagging on the single-classification text set to obtain a part-of-speech-tagged text set; wherein, the single-classification text set can be understood as a text dataset of the same target domain (i.e., positive class domain), which can be obtained according to the actual application scenario, and is not specifically limited here; the part-of-speech-tagged text set can be understood as a single-classification text set with part-of-speech tagging results obtained by tagging the part-of-speech words of interest in each training text sample in the single-classification text set. The part-of-speech words of interest can be arbitrarily determined according to the application requirements in principle, but considering that the content words in the text are often related to the knowledge domain, while function words can be used in any different domain, that is, when judging the domain of the text, it is only necessary to focus on the content words in the sentence, such as the application characteristics of nouns, proper nouns and verbs, in this embodiment, preferably, the target of fine-grained semantic analysis of the samples is positioned on three parts of speech: nouns (NN), proper nouns (NR) and verbs (VV), and the single-classification text set is tagged with three types of parts of speech accordingly; specifically, the step of performing part-of-speech tagging on the single-classification text set to obtain the part-of-speech-tagged text set includes:

[0060] Based on the preset target parts of speech and the FastHan tool, the training text samples in the single-classification text set are labeled with parts of speech to obtain the labeled text set; the preset target parts of speech include nouns, verbs and proper nouns; it should be noted that in practical applications, the range of preset target parts of speech can be adjusted according to the actual application scenario requirements without affecting the application effect of the method of the present invention.

[0061] S12. Based on the part-of-speech tagging text set, construct multiple part-of-speech autoencoders, and based on each part-of-speech autoencoder, obtain the strong semantic words of the part-of-speech in the single-classification text set; wherein, the part-of-speech autoencoder includes a noun autoencoder, a verb autoencoder, and a proper noun autoencoder, which can be understood as an autoencoder used to capture the association between a specific part of speech and the target domain. It can guide the model to select or combine features, identify important features, and learn their probability distribution by training by compressing the original high-dimensional features to a low-dimensional feature space and then restoring them; in this embodiment, the part-of-speech autoencoder is used to filter out strong semantic words that are highly relevant to the target domain, which can more accurately describe the domain affiliation of the positive class, and is the key basis for generating difficult negative samples in this embodiment;

[0062] Specifically, the step of constructing multiple part-of-speech autoencoders based on the part-of-speech-tagged text set includes:

[0063] The part-of-speech-tagged text set is input into the pre-trained model for word meaning extraction to obtain the corresponding word semantic feature set. The pre-trained model can be understood as a network model that can be trained on Chinese corpora. In this embodiment, the RoBERTaChinese model is preferably used to extract the word meaning representation of the text samples. It should be noted that each word semantic feature (word meaning representation) in the word semantic feature set can be understood as first obtaining the hidden vector of each token (character) in the text sample, and then averaging the hidden vectors of the last layer of all tokens contained in the word to obtain the semantic representation of the corresponding word. Figure 4 As shown, for Figure 2 The training text sample shown is "The men's basketball team of country A has become 'team of country B.' Rare data reveals a transformation." When the entire text is input into the RoBERTaChinese model, each token—a character and punctuation mark—corresponds to a contextual semantic representation h. i (i = 1…23), where the semantic representation of the word "men's basketball" is represented by the average of the semantic representations of the two corresponding tokens, namely h3 and h4; in addition, it should be noted that the pre-trained model is a self-attention-based model, and the semantics of each word it outputs not only include the original semantics of the word, but also the semantics of the context. For example, the semantic features of the word "men's basketball" output by the pre-trained model are not completely equal in two different sentences.

[0064] Based on the part-of-speech category, the semantic feature set of the words is classified to obtain multiple part-of-speech semantic feature sets; wherein, the part-of-speech category can be understood as the different part-of-speech types included in the above-mentioned preset target part-of-speech, that is, consistent with the content of the preset target part-of-speech; the corresponding part-of-speech semantic feature sets also include noun part-of-speech semantic feature sets, verb part-of-speech semantic feature sets, and proper noun part-of-speech semantic feature sets, which can be understood as a set of representations of all words in the part-of-speech annotated text set that are annotated as nouns, verbs, and proper nouns, which are integrated with the contextual semantics;

[0065] Each part-of-speech semantic feature set is used to train a pre-defined autoencoder to obtain a corresponding part-of-speech autoencoder. Specifically, the pre-defined autoencoder is trained separately based on the noun part-of-speech semantic feature set, the verb part-of-speech semantic feature set, and the proper noun part-of-speech semantic feature set to obtain corresponding noun autoencoders, verb autoencoders, and proper noun autoencoders. In this embodiment, a pre-defined autoencoder with four fully connected layers is preferably used. Specifically, the step of training the pre-defined autoencoder according to each part-of-speech semantic feature set to obtain the corresponding part-of-speech autoencoder includes:

[0066] Based on the part-of-speech semantic feature set, the preset autoencoder is trained using the reconstruction mean squared error loss function to obtain the corresponding part-of-speech autoencoder; the reconstruction mean squared error loss function is expressed as:

[0067] L = argmin||X - dec[enc(X)]|| 2 (1)

[0068] Where L represents the mean squared error loss of reconstruction; X represents the part-of-speech semantic features in the part-of-speech semantic feature set; enc(X) and dec(enc(X)) represent the encoded features and the corresponding decoded reconstructed features of the part-of-speech semantic features X obtained by the preset autoencoder, respectively;

[0069] In practical applications, the specific training process of part-of-speech autoencoders can be understood as follows: inputting the contextual semantic representations (part-of-speech semantic feature sets) of all words of the same part of speech into a 4-layer fully connected autoencoder, reducing the dimensionality to a low-dimensional hidden layer, and then reconstructing it. The mean squared error loss function is then reconstructed according to equations (2)-(4) to train autoencoders AE for nouns, verbs, and proper nouns respectively. NN AE VV and AE NR :

[0070]

[0071]

[0072]

[0073] Among them, X NN X NR and X VV represents the part-of-speech semantic features of a single noun, proper noun, and verb, respectively; enc represents the encoding part of the autoencoder that compresses the contextual semantics of the word from a higher dimension (768) to a lower dimension (300), and dec represents the decoding part of the autoencoder that restores the original word semantics from the low-dimensional latent space representation to a high-dimensional representation.

[0074] In this embodiment, the loss function used for autoencoder training is the mean squared error between the original sample and the reconstructed sample. This training objective can guide the autoencoder to give greater weight to the common and significant features of the training samples, thereby reducing the mean squared error of the reconstruction. In the single-class text set, all or most samples are related to the positive domain. Therefore, among the nouns, proper nouns, and verbs in the samples, the majority are highly correlated with the positive category. Thus, the autoencoder training method provided in this embodiment can effectively guide the model to focus on the features of words of the three parts of speech that are highly correlated with the positive category, thereby improving the accuracy of subsequent acquisition of strong semantic words.

[0075] After obtaining the noun autoencoder, verb autoencoder, and proper noun autoencoder through the above methods and steps, the part-of-speech semantic features of any noun, verb, or proper noun can be input into the corresponding part-of-speech autoencoder according to the following method. Based on the principle that the smaller the mean square error before and after training, the more features the word has that are related to the positive domain, that is, the greater its contribution to the overall sentence semantic determination to belong to the positive domain, the more likely it is to be a strong semantic word, effective strong semantic words are screened out; specifically, the step of obtaining the strong semantic words of the single-classification text set according to each part-of-speech autoencoder includes:

[0076] Each word semantic feature in each part-of-speech semantic feature set is input into the corresponding part-of-speech autoencoder for feature reconstruction, and the corresponding word reconstruction mean square error value is obtained.

[0077] The mean squared error values ​​of the reconstruction of each word in the same part-of-speech semantic feature set are sorted in ascending order, and words of a preset proportion are selected from top to bottom as the corresponding strong semantic words of the part of speech. The preset proportion (which is a hyperparameter) can be selected according to the actual application requirements. The principle is that the words ranked first in the ascending order of the mean squared error values ​​of the reconstruction of each word in the same part-of-speech semantic feature set are more related to the target domain. For example, if the preset proportion is set to a fixed proportion of 70% through experiments or experience, then the 70% of words with the smallest mean squared error values ​​of reconstruction of the same part of speech will be labeled as strong semantic words of this part of speech.

[0078] S13. Replace all strongly semantically related words in the single-classification text set with preset semantically neutral words of equal length to obtain the corresponding weakly semantically difficult sample set. The weakly semantically difficult sample set can be understood as a set of weakly semantically difficult negative samples obtained by replacing all strongly semantically related words in the single-classification text set with words representing semantically neutral words in the pre-trained model, based on the consideration that strongly semantically related words have a high degree of domain relevance and contribute significantly to the text domain attribution judgment. Correspondingly, the preset semantically neutral words can be selected according to actual application needs, for example, replacing each character in the strongly semantically related words with the string "[PAD]" to generate a sample set like... Figure 5 The weak semantically difficult negative samples shown;

[0079] It should be noted that in existing text single-class classification and out-of-domain sample recognition problems, negative samples mainly come from three sources: external datasets, GAN (Generative Adversarial Network) based methods, and Mixup based methods. Each of these methods has varying degrees of limitations: methods using external datasets primarily introduce simple negative samples. When the negative samples have high similarity to positive samples, they are considered difficult negative samples, which can improve the contrastive learning effect to some extent. However, when some samples in the external dataset belong to the positive domain, they can introduce noise into the model training, reducing the model's learning effect. GAN-based and Mixup-based methods target the entire sentence semantics, without utilizing the semantics of fine-grained words within the sentence, making it difficult to achieve high similarity with positive samples, and the difficulty of generating negative samples is limited. The weak semantic difficult sample generation method provided in this embodiment can retain features unrelated to the positive domain in the original sentence to the maximum extent, ensuring high similarity with the original samples while accurately reducing the positive domain relevance of the samples. This results in a higher difficulty in generating negative samples and further promotes the training effect of contrastive learning.

[0080] S14. Based on the single-classification text set and the weak semantic hard sample set, a pre-trained model is trained using category-level supervised contrastive learning to obtain a domain feature extractor. Based on the domain feature extractor, a domain feature prototype corresponding to the single-classification text set is constructed. Supervised contrastive learning can be understood as training the feature extractor using InfoNCE or similar loss functions to map samples to a latent feature space. In this latent feature space, positive samples cluster within a specific range, are sufficiently compact, while negative samples are distributed outside this specific range, and all negative samples maintain a certain distance from the specific range of positive samples. Simultaneously, during the specific training process, by continuously inputting contrastive learning of positive and negative samples, the feature extractor is guided to give more attention weight to features shared by positive samples and less attention weight to features shared or similar between positive and negative samples, in order to identify features related to the domain of positive samples.

[0081] In practical applications, the construction of a contrastive learning feature extractor can be achieved either by training from scratch or by fine-tuning a pre-trained model. However, considering the powerful fitting ability of the pre-trained model shown above in acquiring text semantics, this embodiment preferably uses the Chinese-Roberta-WWM-Ext fine-tuning method, combined with backpropagation gradient updates, to update the pre-trained model parameters to obtain the optimal domain feature extractor; specifically, as shown... Figure 6 As shown, the step of training a pre-trained model using category-level supervised contrastive learning based on the single-classification text set and the weak semantic difficulty sample set to obtain a domain feature extractor includes:

[0082] The pre-trained model is initialized based on the network structure and parameters of the chinese-roberta-wwm-ext model;

[0083] The single-class text set and the weak semantic difficulty sample set are respectively designated as the positive class sample set and the negative class sample set; wherein, the positive class sample set can be understood as the set obtained by labeling all samples in the single-class text set with positive class labels, and correspondingly, the negative class sample set can be understood as the set obtained by labeling all samples in the weak semantic difficulty sample set with negative class labels; the aforementioned positive class sample set and negative class sample set can be understood as the training data for the following class-level supervised contrastive learning;

[0084] Using each positive sample in the positive sample set as an anchor sample, and other positive samples in the same batch as positive samples, and simultaneously selecting the same number of weak semantic negative samples as the positive samples in the negative sample set, the pre-trained model is backpropagated and trained according to a preset loss function to obtain the domain feature extractor; wherein, the preset loss function is preferably the infoNCE loss function, specifically expressed as:

[0085]

[0086] In the formula,

[0087] z i =h(x i ) [cls]

[0088] Among them, L contrastive N represents the infoNCE loss function for supervised contrastive learning; p Indicates the number of positive samples; Let x represent the discriminant function, that is, let zi and zj be the domain features corresponding to any two different samples in the training set. The domain features can be understood as each text sequence x. i After the training samples (from the original training set or the generated hard negative samples) are input into the domain feature extractor h, according to the characteristics of the BERT series models, the last latent vector z corresponding to [CLS] is selected. i The feature vector of the entire text sequence xi with respect to the positive domain; N represents the batch size (number of training batches); This means taking all domain features of all other samples within the entire batch, including all positive samples and all negative samples; sim() represents the calculation of the similarity of domain features between two samples, preferably using cosine similarity; T represents the temperature coefficient, used to adjust the model's attention to difficult negative samples;

[0089] It should be noted that the specific construction process of the domain feature extractor in this embodiment can be understood as using the network structure and parameters of the chinese-roberta-wwm-ext model as the initial state of the domain feature encoder, and calculating the training loss L by continuously inputting samples. contrastive The updated echelon is then backpropagated using backpropagation, with the current parameter value plus the gradient value used as the updated parameter value. Through this continuous training and parameter updating, the domain feature encoder moves closer to the positive sample encoding in the feature space, while the feature encoding of the generated difficult negative samples moves further away from the positive sample group. This is an optimized class-level supervised contrastive learning approach. Furthermore, compared with existing individual sample-level contrastive learning methods (i.e., positive sample pairs are anchor samples and their data augmentation results, while other samples and their data augmentation results are considered negative samples), the class-level supervised contrastive learning adopted in this embodiment utilizes category features to construct sample pairs based on categories. That is, all samples in the original training set are considered positive sample pairs to each other, while the generated negative samples are considered negative sample pairs to each other with positive samples. This effectively increases the aggregation degree of samples of the same category in the feature space, thereby improving the training effect of contrastive learning.

[0090] The domain feature prototype can be understood as a standard used to determine the target domain of the sample to be identified, that is, to determine whether the sample to be identified belongs to an outside domain. Considering that the correlation between each positive class sample in the single-class text set and the domain is different, it is easy to have deviations when calculating the similarity between the domain features of a single positive class sample and the domain features of the sample to be identified. In order to obtain a feature prototype that can accurately represent the target domain, this embodiment preferably uses the average value of the domain features of all positive class samples in the single-class text set as the domain feature prototype. Specifically, the step of constructing the domain feature prototype corresponding to the single-class text set according to the domain feature extractor includes:

[0091] Each positive class sample in the single-class text set is input into the domain feature extractor for feature extraction to obtain the corresponding domain features; wherein, the domain features can be understood as each positive class sample x i The input is fed into the domain feature extractor, and the output is the last latent vector h'(x) corresponding to the [CLS] label. i ) [cls] ;

[0092] Calculate the average value of all domain features to obtain the domain feature prototype; wherein, the domain feature prototype is represented as:

[0093]

[0094] In the formula, h'(x i ) [cls] Represents positive class samples x in a single-class text set.i Domain characteristics; N represents the number of all positive class samples in the single-class text set; z prototype Represents the prototype of domain features.

[0095] S15. Input the text to be identified into the domain feature extractor for feature extraction to obtain the corresponding domain features to be identified, and calculate the similarity between the domain features to be identified and the domain feature prototype to obtain the corresponding domain score of the text to be identified; wherein, the domain features to be identified can be understood as the last latent vector z corresponding to the [CLS] label output after the text to be identified is processed by the domain feature extractor. test =h'(x test ) [cls] Correspondingly, the domain score of the text to be identified is represented as:

[0096] S ind =sim(z) test , z prototype (7)

[0097] Among them, S ind This indicates the domain score of the text to be identified.

[0098] S16. Based on the domain score of the text to be identified and the preset scoring threshold, obtain the corresponding classification result of the text to be identified; wherein, the preset scoring threshold can be determined according to the actual application requirements, and the corresponding classification result of the text to be identified includes in-domain samples and out-of-domain samples; in actual application, if the domain score of the text to be identified is less than the preset scoring threshold, the text to be identified is determined to be an out-of-domain sample, otherwise, the text to be identified is determined to be an in-domain sample.

[0099] This application embodiment obtains a part-of-speech (POS) labeled text set by performing part-of-speech tagging on the acquired single-class text set. Multiple POS autoencoders are constructed based on the POS labeled text set, and strong semantic words of the single-class text set are obtained from each POS autoencoder. After replacing all strong semantic words in the single-class text set with pre-defined semantically neutral words of equal length to obtain a corresponding weak semantically difficult sample set, a pre-trained model is trained using category-level supervised contrastive learning based on the single-class text set and the weak semantically difficult sample set to obtain a domain feature extractor. A domain feature prototype corresponding to the single-class text set is constructed based on the domain feature extractor. Finally, after inputting the text to be identified into the domain feature extractor for feature extraction to obtain the corresponding domain features to be identified, the text to be identified is then processed. The scheme calculates the similarity between domain features and domain feature prototypes to obtain the corresponding domain score of the text to be identified. Based on the domain score of the text to be identified and a preset scoring threshold, the corresponding classification result of the text to be identified is obtained. Through a contrastive learning method based on weak semantic negative samples, the scheme deeply explores the association between fine-grained words of various parts of speech and the domain to which the text belongs. Data augmentation is performed on the original samples to generate difficult negative samples with high similarity to the original samples but low domain relevance. Then, contrastive learning is used to fine-tune the pre-trained model to obtain a better domain feature extractor. This achieves the goal of being able to identify samples outside the domain by directly calculating the similarity between the domain features of the text to be identified and the domain prototype features. This greatly improves the training effect of contrastive learning and thus effectively improves the classification accuracy of single-class text.

[0100] Furthermore, to further illustrate the application effect of the text single-class classification method of the present invention, this embodiment also sets up a comparative verification with existing text single-class classification methods in a text single-class classification scenario, in terms of the best recognition effect on out-of-domain samples, the recognition effect on different numbers of training samples, and the recognition effect on scenarios where training samples are contaminated, as detailed below:

[0101] Evaluation metrics: Each input sample corresponds to an evaluation score. Different thresholds will yield different evaluation results. Therefore, this experiment uses AUC (area under the ROC curve of the receiver operating characteristic curve) as the main evaluation metric. In addition, based on the threshold on the AUC curve where the TPR is the largest and the FPR is the smallest, the accuracy ACC and macro-F1 (F1 value sensitive to the number of classes) of the model under this threshold are used as auxiliary evaluation metrics.

[0102] Dataset: This experiment primarily uses the THUNEWS news classification dataset. This dataset was compiled from data of a subscription channel from 2005 to 2011, filtering and integrating it to divide it into 14 candidate categories, totaling 830,000 records. The experiment uses 8,000 samples from a randomly selected category as the training set for single-class classification. A validation set is created by mixing 20,000 positive samples from the same category from the remaining data with 20,000 negative samples randomly selected from other categories. In experiments examining the impact of different numbers of training samples on the training results, samples are randomly drawn from the existing single-class training set according to the required number. In experiments investigating training set contamination, negative samples are drawn and added to the training set at different ratios. In the experiment comparing with the baseline model, another Chinese dataset, TNEWS (Chinese News (Short Text) Classification Dataset), was also used, extracting news from 15 categories, including tourism, education, finance, and military. Similarly, 8,000 samples from one of the 15 categories are randomly selected as the intra-domain training set for single-class classification. Then, 20,000 positive samples from the same category and 20,000 negative samples from other categories are randomly selected from the remaining data to form the validation set.

[0103] 1) Verification of optimal recognition performance

[0104] like Figure 7 As shown, with a training sample size of 8000, compared with existing methods, the method of this invention achieves the best performance in identifying out-of-domain text single-class classification samples on both databases. Compared with the second-ranked method DAE, on the THUNEWS dataset, the method of this invention improves the AUC score by 0.0127, the ACC by 0.0355, and the F1 macro by 0.0367; on the TNEWS dataset, the method of this invention improves the AUC score by 0.0079, the ACC by 0.0288, and the F1 macro by 0.0488.

[0105] 2) Verification of recognition performance with different numbers of training samples

[0106] like Figure 8 As shown, the method of this invention achieves the best results under different training set settings (x-axis represents the number of training samples), and also performs well in scenarios with few samples. It can even obtain an AUC value of 0.9640, a highest accuracy of 0.9147, and a highest F1_macro value of 0.9113 using only 3 training samples. This is a significant improvement over the second-ranked DAE (AUC improved by 0.0352, ACC improved by 0.0505, and F1_macro improved by 0.0530). Moreover, the model can achieve near-optimal results (AUC = 0.9831) with approximately 100 samples for training.

[0107] 3) Verification of recognition performance in scenarios where training samples are contaminated.

[0108] To further verify the robustness of the method of the present invention, this embodiment also attempts to mix a certain proportion of negative samples into the training set while maintaining a sample size of 800, ranging from 1% to 70%; for example... Figure 9 As shown, although the effectiveness decreases to some extent with the increase of the negative sample ratio, the method of the present invention is always superior to the other existing methods. Moreover, it still has an AUC value of 0.8794 with a negative sample ratio of 50%, which is 0.06 higher than the second-ranked SVDD. In addition, the effectiveness of the method of the present invention decreases more slowly than that of SVDD as the negative sample ratio continues to increase, indicating that the method of the present invention has strong robustness.

[0109] Based on the above comparative verification results, it can be seen that this invention not only makes up for the gap in current text single-class classification research that has not sufficiently fine-tuned the pre-trained model and has not further explored the relationship between fine-grained words and domains, effectively improving the ability to identify samples outside the domain in single-class classification, but also has strong small-sample learning ability and robustness.

[0110] It should be noted that although the steps in the flowchart above are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order requirement for the execution of these steps, and they can be executed in other orders.

[0111] In one embodiment, such as Figure 10 As shown, a text single-category classification system is provided, the system comprising:

[0112] Preprocessing module 1 is used to obtain a single-classification text set and perform part-of-speech tagging on the single-classification text set to obtain a part-of-speech tagged text set;

[0113] The strong semantic word acquisition module 2 is used to construct multiple part-of-speech autoencoders based on the part-of-speech labeled text set, and to acquire the strong semantic words of the part-of-speech in the single-classification text set based on each part-of-speech autoencoder;

[0114] Negative sample generation module 3 is used to replace all strongly semantic words in the single-classification text set with preset words of equal length without semantic meaning to obtain the corresponding weak semantic difficult sample set.

[0115] The contrastive learning training module 4 is used to perform category-level supervised contrastive learning training on the preset pre-trained model based on the single-classification text set and the weak semantic difficulty sample set, to obtain a domain feature extractor, and to construct the domain feature prototype corresponding to the single-classification text set based on the domain feature extractor.

[0116] Domain scoring module 5 is used to input the text to be identified into the domain feature extractor for feature extraction, obtain the corresponding domain features to be identified, and calculate the similarity between the domain features to be identified and the domain feature prototype to obtain the corresponding domain score of the text to be identified.

[0117] Result generation module 6 is used to obtain the corresponding classification result of the text to be identified based on the domain score of the text to be identified and the preset score threshold.

[0118] Specific limitations regarding the text single-class classification system can be found in the limitations of the text single-class classification method described above, and the corresponding technical effects can be obtained equivalently, so they will not be repeated here. Each module in the above-mentioned text single-class classification system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0119] Figure 11 An internal structural diagram of a computer device is shown in one embodiment. This computer device may specifically be a terminal or a server. Figure 11 As shown, the computer device includes a processor, memory, network interface, display, camera, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a single-category text classification method. The display screen can be an LCD screen or an e-ink display screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0120] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computing devices may include more or fewer components than those shown in the figure, or combine certain components, or have the same component arrangement.

[0121] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0122] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0123] In summary, the single-class text classification method and system provided by this invention involves: performing part-of-speech tagging on an acquired single-class text set to obtain a tagged text set; constructing multiple part-of-speech autoencoders based on the tagged text set; obtaining strong semantic words from the single-class text set based on each autoencoder; replacing all strong semantic words in the single-class text set with pre-defined semantically neutral words of equal length to obtain a corresponding weak semantically difficult sample set; training a pre-trained model using category-level supervised contrastive learning based on the single-class text set and the weak semantically difficult sample set to obtain a domain feature extractor; constructing a domain feature prototype corresponding to the single-class text set based on the domain feature extractor; and inputting the text to be identified into the domain feature extractor for feature extraction to obtain the corresponding domain features to be identified, and then comparing the domain features to be identified with... This method employs a domain feature prototype similarity calculation to obtain the corresponding domain score for the text to be identified. Based on the domain score and a preset scoring threshold, the corresponding classification result of the text to be identified is obtained. This method uses a contrastive learning approach based on weak semantic negative samples to deeply explore the association between fine-grained words of various parts of speech and the domain to which the text belongs. It performs data augmentation on the original samples to generate difficult negative samples with high similarity to the original samples but low domain relevance. Then, it uses contrastive learning to fine-tune the pre-trained model to obtain a better domain feature extractor. This achieves the goal of being able to identify samples outside the domain by directly calculating the similarity between the domain features of the text to be identified and the domain prototype features. This not only greatly improves the training effect of contrastive learning and effectively enhances the ability to identify samples outside the domain in single-class classification, but also improves the classification accuracy of single-class text. Furthermore, it has strong small-sample learning ability and robustness.

[0124] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0125] The embodiments described above are merely preferred embodiments of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the technical principles of this invention, and these improvements and substitutions should also be considered within the scope of protection of this application. Therefore, the scope of protection of this patent application should be determined by the scope of the claims.

Claims

1. A text single-category classification method, characterized in that, The method includes the following steps: Obtain a single-classification text set and perform part-of-speech tagging on the single-classification text set to obtain a part-of-speech tagged text set; Based on the part-of-speech-tagged text set, multiple part-of-speech autoencoders are constructed, and based on each part-of-speech autoencoder, strong semantic words of the part-of-speech in the single-classification text set are obtained; the part-of-speech autoencoders include noun autoencoders, verb autoencoders, and proper noun autoencoders; Replace all strongly semantic words in the single-classification text set with preset words of equal length that have no semantic meaning to obtain the corresponding weak semantic difficult sample set. Based on the single-class text set and the weak semantic hard sample set, a pre-trained model is trained by category-level supervised contrastive learning to obtain a domain feature extractor, and based on the domain feature extractor, a domain feature prototype corresponding to the single-class text set is constructed. The text to be identified is input into the domain feature extractor for feature extraction to obtain the corresponding domain features to be identified. The similarity between the domain features to be identified and the domain feature prototype is calculated to obtain the corresponding domain score of the text to be identified. Based on the domain score of the text to be identified and the preset scoring threshold, the corresponding classification result of the text to be identified is obtained; The step of constructing multiple part-of-speech autoencoders based on the part-of-speech-tagged text set includes: The part-of-speech-tagged text set is input into the pre-trained model for word meaning extraction to obtain the corresponding word semantic feature set; Based on the part-of-speech category, the semantic feature set of the words is classified to obtain multiple part-of-speech semantic feature sets; The preset autoencoder is trained according to each part-of-speech semantic feature set to obtain the corresponding part-of-speech autoencoder; the preset autoencoder includes 4 fully connected layers; The step of training the preset autoencoder based on each part-of-speech semantic feature set to obtain the corresponding part-of-speech autoencoder includes: Based on the part-of-speech semantic feature set, the preset autoencoder is trained using the reconstruction mean squared error loss function to obtain the corresponding part-of-speech autoencoder; the reconstruction mean squared error loss function is expressed as: Where L represents the mean squared error loss of reconstruction; X represents the part-of-speech semantic features in the part-of-speech semantic feature set; enc(X) and dec(enc(X)) represent the encoded features and the corresponding decoded reconstructed features obtained by the part-of-speech semantic features X through a preset autoencoder, respectively.

2. The text single-category classification method as described in claim 1, characterized in that, The step of performing part-of-speech tagging on the single-classified text set to obtain a part-of-speech tagged text set includes: Based on the preset target parts of speech and the FastHan tool, the training text samples in the single-classification text set are labeled with parts of speech to obtain the part-of-speech labeled text set; the preset target parts of speech include nouns, verbs and proper nouns.

3. The text single-category classification method as described in claim 1, characterized in that, The step of obtaining strongly semantic words with part-of-speech tags from the single-classification text set based on each part-of-speech autoencoder includes: Each word semantic feature in each part-of-speech semantic feature set is input into the corresponding part-of-speech autoencoder for feature reconstruction, and the corresponding word reconstruction mean square error value is obtained. The mean squared error values ​​of the reconstruction of each word in the same part-of-speech semantic feature set are arranged in ascending order, and words of a predetermined proportion are selected from top to bottom as the corresponding strong part-of-speech semantic words.

4. The text single-category classification method as described in claim 1, characterized in that, The step of training a pre-trained model using category-level supervised contrastive learning based on the single-classification text set and the weak semantic difficulty sample set to obtain a domain feature extractor includes: The pre-trained model is initialized based on the network structure and parameters of the chinese-roberta-wwm-ext model; The single-classification text set and the weak semantic difficulty sample set are respectively used as the positive class sample set and the negative class sample set; Each positive sample in the positive sample set is used as an anchor sample, and other positive samples in the same batch are used as positive samples. Then, weak semantic negative samples with the same number as the positive samples are selected in the negative sample set. The pre-trained model is then trained by backpropagation according to a preset loss function to obtain the domain feature extractor.

5. The text single-category classification method as described in claim 1, characterized in that, The step of constructing the domain feature prototype corresponding to the single-classification text set based on the domain feature extractor includes: Each positive class sample in the single-class text set is input into the domain feature extractor for feature extraction to obtain the corresponding domain features; Calculate the average value of all domain features to obtain the domain feature prototype.

6. A text single-category classification system, characterized in that, The system, employing the text single-category classification method as described in claim 1, comprises: The preprocessing module is used to obtain a single-classification text set and perform part-of-speech tagging on the single-classification text set to obtain a part-of-speech tagged text set; The strong semantic word acquisition module is used to construct multiple part-of-speech autoencoders based on the part-of-speech labeled text set, and to acquire the strong semantic words of the part-of-speech in the single-classification text set based on each part-of-speech autoencoder; The negative sample generation module is used to replace all strongly semantic words in the single-classification text set with preset words of equal length without semantic meaning to obtain the corresponding weak semantic difficult sample set. The contrastive learning training module is used to perform category-level supervised contrastive learning training on a preset pre-trained model based on the single-classification text set and the weak semantic difficulty sample set, to obtain a domain feature extractor, and to construct a domain feature prototype corresponding to the single-classification text set based on the domain feature extractor. The domain scoring module is used to input the text to be identified into the domain feature extractor for feature extraction, obtain the corresponding domain features to be identified, and calculate the similarity between the domain features to be identified and the domain feature prototype to obtain the corresponding domain score of the text to be identified. The result generation module is used to obtain the corresponding classification result of the text to be identified based on the domain score of the text to be identified and the preset score threshold.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images