Supervised Text Classification Method and System Based on Virtual Adversarial and Bidirectional Contrastive Learning

Abstract

The present invention belongs to the field of text classification, and provides a supervised text classification method and system based on virtual adversarial and bidirectional contrast learning. Based on the framework of virtual adversarial and bidirectional contrast learning, this framework combines virtual adversarial training and contrast learning, so that the model can be trained to establish a better relationship between adversarial samples and real samples, improving the robustness and generalization performance of the classification model. Then, enhanced samples are generated using the label information of the samples, and bidirectional contrast learning is used to learn the relationship between the sample labels and features, further improving the accuracy of the classification model. Finally, the VABCL framework is experimentally verified on three benchmark text classification datasets. The experimental results confirm the improvement of the feature extraction ability and classification accuracy of VABCL, indicating that VABCL has good performance in the field of supervised text classification.

Description

Technical Field

[0001] The present invention belongs to the field of text classification, and particularly relates to a supervised text classification method and system based on virtual adversarial and bidirectional contrast learning. Background Art

[0002] The statements in this part only provide background technical information related to the present invention, and do not necessarily constitute prior art.

[0003] The existing text classification methods have the following technical problems:

[0004] 1. There are problems such as poor generalization in supervised text classification for contrast learning. For example, in the same dataset, dual contrast learning has the best effect on the bert model compared to other methods, but it is not the optimal method on RoBerta, and compared with unsupervised learning, it performs poorly in larger batches and longer training.

[0005] 2. The existing contrast learning techniques cannot learn the information between samples, and the label information cannot be effectively utilized. And in the experiment, the model still needs to learn a linear classifier using cross-entropy loss in addition to the contrast items.

[0006] 3. The computational cost of traditional adversarial training is high. A large number of adversarial samples need to be constructed, which will increase the computational cost and lead to longer training time. And the generalization ability of traditional adversarial training is limited. The purpose of adversarial training is to improve the robustness of the model, but traditional adversarial training cannot guarantee the generalization ability of the model in real scenarios. This is because the construction method of adversarial samples may be different from the attack method in real scenarios, resulting in the model being unable to effectively cope with attacks in real scenarios. Summary of the Invention

[0007] In order to solve at least one of the above technical problems in the background art, the present invention provides a supervised text classification method based on virtual adversarial and bidirectional contrast learning, which combines virtual adversarial training and contrast learning, so as to train the model to establish a better relationship between adversarial samples and real samples, and improve the robustness and generalization performance of the classification model. Then, enhanced samples are generated using the label information of the samples, and bidirectional contrast learning is used to learn the relationship between sample labels and features, further improving the accuracy of the classification model.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] The first aspect of the present invention provides a supervised text classification method based on virtual adversarial and bidirectional contrast learning, including the following steps:

[0010] Obtain text sample data and corresponding label information;

[0011] Train a text classification model based on text sample data and corresponding label information to obtain a trained text classification model; wherein, the construction method of the text classification model includes:

[0012] Combine virtual adversarial training and contrastive learning, calculate the perturbation using the calculation method of the best perturbation in virtual adversarial training, combine the calculated perturbation with the text sample data to generate adversarial text data for contrastive learning, and in two contrastive learning processes, through a bidirectional contrastive loss function, simultaneously learn the feature representation of the adversarial text data and its associated class label.

[0013] Classify the text based on the trained text classification model to obtain a text classification result.

[0014] The second aspect of the present invention provides a supervised text classification system based on virtual adversarial and bidirectional contrastive learning, including:

[0015] A data acquisition module, which is used to acquire text sample data and corresponding label information;

[0016] A classification model training module, which is used to train a text classification model based on text sample data and corresponding label information to obtain a trained text classification model; wherein, the construction method of the text classification model includes:

[0017] Combine virtual adversarial training and contrastive learning, calculate the perturbation using the calculation method of the best perturbation in virtual adversarial training, combine the calculated perturbation with the text sample data to generate adversarial text data for contrastive learning, and in two contrastive learning processes, through a bidirectional contrastive loss function, simultaneously learn the feature representation of the adversarial text data and its associated class label.

[0018] A text classification module, which is used to classify the text based on the trained text classification model to obtain a text classification result.

[0019] The third aspect of the present invention provides a computer-readable storage medium.

[0020] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the supervised text classification method based on virtual adversarial and bidirectional contrastive learning as described in the first aspect.

[0021] The fourth aspect of the present invention provides a computer device.

[0022] A computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, it implements the steps in the supervised text classification method based on virtual adversarial and bidirectional contrastive learning as described in the first aspect.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] 1. The present invention introduces virtual adversarial training in contrastive learning and applies it to supervised learning. First, a virtual adversarial pre-trained model is used, and then double contrastive learning is used for fine-tuning. The model designs two contrastive learnings, which can simultaneously learn the feature representation of the sample and the output of the classifier. The output of the classifier is a data augmentation sample set with labels, so that the model can be trained to establish a better relationship between adversarial samples and real samples, improving the robustness and generalization performance of the classification model.

[0025] 2. The two contrastive learning loss functions designed by the present invention can effectively utilize label information to classify text and can learn the information between samples, solving the defect that standard contrastive learning cannot utilize label information and further improving the accuracy of the classification model.

[0026] 3. The virtual adversarial training of the present invention can solve the problems of insufficient generalization and robustness of the current pre-trained model, and solve the problem that although adversarial training can enhance robustness, it will damage generalization.

[0027] Advantages of additional aspects of the present invention will be partially given in the following description, partially become apparent from the following description, or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0029] Figure 1 It is a flowchart of supervised text classification based on virtual adversarial and bidirectional contrastive learning of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0031] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0032] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should also be understood that when the terms "comprise" and / or "include" are used in this specification, they specify the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0033] Term Explanation

[0034] Contrastive learning: A type of unsupervised learning where unlabeled data is given and the model learns a feature representation on its own. In short, contrastive learning requires the neural network to automatically learn a feature representation such that similar instances are close in the projection space and dissimilar instances are far apart in the projection space, thus enabling text classification.

[0035] The applications of contrastive learning in text classification can be mainly divided into two categories: applications in supervised and unsupervised text classification. In supervised text classification, contrastive learning does not perform very well. For example, SCL (Supervised Contrastive Learning) and Dual Contrastive Learning (Dualcl). The contrastive loss function in SCL presents a novel extension that allows multiple positive samples for each anchor point, adapting contrastive learning to the supervised scenario. Compared with general models, the accuracy has been improved on multiple datasets.

[0036] Dual contrastive learning can improve the performance of text classification. Especially when dealing with high-dimensional sparse data, by optimizing the difference between positive and negative samples, dual contrastive learning can learn a more robust embedding representation, thereby improving the performance of text classification.

[0037] Virtual Adversarial Training (VAT): A regularization algorithm improved based on adversarial training. Virtual adversarial training is superior to traditional adversarial training and is applicable to both pre-training and fine-tuning, especially when the labels may be noisy. Virtual adversarial training aims to find a perturbation direction that maximizes the output deviation. Perturb the input in this direction and then use it to train the model to enhance the local smoothness of the model. Virtual adversarial training makes the model more robust by generating perturbations automatically by the model, enabling the model to better adapt to unknown and noise-interfered input data. Compared with traditional adversarial training, virtual adversarial training does not require generating adversarial samples but estimates adversarial perturbations by calculating gradients. Therefore, virtual adversarial training has higher robustness, higher efficiency, and a wider range of applications. Both virtual adversarial training and contrastive learning are common techniques to improve model performance, and they can be combined to further improve the accuracy of text classification.

[0038] In the context of unsupervised learning mentioned in the background art, contrastive learning can effectively learn discriminative representations of text features, thereby improving the performance of text classification. However, in the context of supervised learning, contrastive learning methods cannot fully utilize the label information of samples in text classification models, nor can they well learn the relationship between labels and features among samples.

[0039] The present invention proposes a framework of Virtual Adversarial and Bidirectional Contrastive Learning (VABCL) based on virtual adversarial and bidirectional contrastive learning. This framework combines virtual adversarial training and contrastive learning, so as to train the model to establish a better relationship between adversarial samples and real samples, and improve the robustness and generalization performance of the classification model. Then, enhanced samples are generated using the label information of the samples, and bidirectional contrastive learning is used to learn the relationship between sample labels and features, further improving the accuracy of the classification model. Finally, the VABCL framework is experimentally verified on three benchmark text classification datasets, and the experimental results confirm the improvement of the feature extraction ability and classification accuracy of VABCL, indicating that VABCL has good performance in the field of supervised text classification.

[0040] In order to make good use of contrastive learning methods for text classification in the context of supervised learning, the present invention proposes a text classification model based on virtual adversarial and bidirectional contrastive learning. This model develops a dual contrastive learning method introducing virtual adversarial in a supervised environment, introducing virtual adversarial training in contrastive learning and applying it in supervised learning, that is, first using virtual adversarial to pre-train the model and then using dual contrastive learning for fine-tuning. The model designs two contrastive learning processes, which can simultaneously learn the feature representation of the sample and the output of the classifier, where the output of the classifier is a set of data-augmented samples with labels. Standard contrastive learning cannot utilize label information, while the two contrastive learning loss functions designed in this model can effectively utilize label information to classify text and can learn the information between samples. Virtual adversarial training can solve the problems of insufficient generalization and robustness of the current pre-trained model, and solve the problem that although adversarial training can enhance robustness, it will damage generalization.

[0041] Embodiment 1

[0042] This embodiment provides a supervised text classification method based on virtual adversarial and bidirectional contrastive learning, including the following steps:

[0043] Step 1: Obtain text data and corresponding label information;

[0044] Step 2: Based on the text data, the corresponding label information, and the trained text classification model, obtain the trained text classification model;

[0045] The text classification model consists of two parts:

[0046] First, use the calculation method of the optimal perturbation in virtual adversarial training to calculate the perturbation, and combine the calculated perturbation with the original sample to generate adversarial samples for contrastive learning. Then, in bidirectional contrastive learning, learn two loss functions, which respectively learn the input representation of the discriminative features of the classification task and the class labels of the samples in an appropriate space.

[0047] Step 3: Classify the text based on the trained text classification model to obtain the text classification result.

[0048] As Figure 1 shown, the specific implementation process is as follows:

[0049] Let \(Z\) i \(\in\mathbb{R}\) d be the feature of the input sample \(X\) i and \(\theta\) i \(\in\mathbb{R}\) d *\(K\) be the class label associated with \(X\) i . The final goal is to learn the normalized representations of \(Z\) i and \(\theta\) i so as to make the softmax transformation of the classifier parameters, which is , consistent with the label of \(X\) i .

[0050] 1. Combination of virtual adversarial training and contrastive learning

[0051] The first step in the combination of virtual adversarial training and contrastive learning is to define the calculation method of the perturbation \(r\), which is the loss function.

[0052] The specific method for calculating the perturbation in virtual adversarial training is as follows:

[0053] Input the sentence information and label information into two identical encoders. Add perturbation to the first encoder to generate perturbed samples; output clean samples from the second encoder, and use the perturbed samples as anchor points to train their relationship with the clean and correct samples.

[0054] The perturbation is calculated as follows:

[0055]

[0056] In the formula, \(x\) i represents the input sample, represents the model parameters after a certain step of training, \(\theta\) represents the model parameters, \(y\) represents the output label, and \(r\) represents the perturbation generated at a certain step.

[0057] Calculate Does not directly use the gradient with respect to the input sample x i of.

[0058] Because obviously when r = 0, the gradient is 0. After performing a Taylor expansion of it with respect to r at 0, the following approximation is found:

[0059]

[0060] where O(r 2 ) is the higher-order infinitesimal of r 2 , and H is the Hessian matrix.

[0061] From the definitions of eigenvalues and eigenvectors, for a fixed norm size of r, when r is the eigenvector corresponding to the largest eigenvalue, r T Hr is the largest. Also, because the norm of r is very small, the subsequent higher-order infinitesimals can be ignored. Correspondingly, also reaches the maximum.

[0062] So:

[0063]

[0064] where represents the unit eigenvector corresponding to the largest eigenvalue of H, and ∈ represents the hyperparameter;

[0065] In this embodiment, in virtual adversarial training, the power iteration method is used to calculate the eigenvector corresponding to the largest eigenvalue of the matrix, that is, a vector d of the same dimension is randomly taken (assuming that when expressing d with eigenvectors, the coefficient of u is not 0), and the following iteration is performed:

[0066]

[0067] In this embodiment, a good approximation of u can be obtained by iterating once. That is:

[0068]

[0069]

[0070] Step 2: Add noise perturbations and obtain the virtual sample set. Before contrastive learning, add noise to the input of the model and perform data augmentation on the original training set to generate the virtual sample set.

[0071] Specifically, in this embodiment, the VAT loss function is adopted, and a simple MLP is used as the generator to generate perturbations. In each training iteration, the perturbations are added to the input of the model.

[0072] Step 3: Use the virtual sample set for contrastive learning. This module introduces virtual adversarial training in contrastive learning to optimize the similarity between the original samples and the samples with perturbations, enhancing the generalization and robustness of the model.

[0073] Define the following contrastive loss L cal :

[0074]

[0075] In the formula, Z i is the feature of the input sample X i . is the feature of the sample after adding perturbations, τ is the temperature factor, A i is the index set of the contrastive samples, Z a is the feature of the negative samples.

[0076] 2. Bidirectional contrastive learning

[0077] This module uses label-aware data augmentation to obtain augmented samples of the training samples, and the augmented samples are generated using label information.

[0078] The specific approach is to integrate all texts and labels and input them into the Bert encoder. Subsequently, the feature representation of the entire text will be obtained. Among them, the special [CLS] token represents the feature of the entire text, and for each label, the corresponding token-level output can be obtained.

[0079] Suppose there are K labels in the input, then the output label set, i.e., the classifier θ ∈ R d*k . Since each label integrates text information, each column of θ i is an augmented sample, and each sample obtains K + 1 augmented samples. After obtaining multiple augmented samples of the samples, these augmented samples can be used for contrastive learning. Taking the binary classification task as an example: CLS is the feature representation of the sample, and POS and NEG (non-POS samples) are the labels of the sample.

[0080] The design idea of positive and negative samples for this contrastive learning is as follows:

[0081] Loss function 1: Determine the true label POS of a sample. Using the feature CLS of this sample as the anchor point, the outputs with all other labels as POS are positive samples, and the labels of non-POS samples are negative samples.

[0082] Loss function 2: Determine the true label POS of a sample. Using the output with this sample label as POS as the anchor point, then the CLS of the outputs with all other sample labels as POS are positive samples, and the CLS of non-POS samples are negative samples.

[0083] Loss function 2: Determine the true label POS of a sample. Using the output with this sample label as POS as the anchor point, then the CLS of the outputs with all other sample labels as POS are positive samples, and the CLS of non-POS samples are negative samples.

[0084] Given an anchor Z from the input example X i , let θ i denote the true label of x i , A be the set of negative samples, take i as positive samples respectively, take as negative samples, and Z as the true feature of the sample. i

[0085] In loss function 1, fix the label θ i , and define the following contrastive loss:

[0086]

[0087] In loss function 2, fix the true feature z i , and define the following contrastive loss:

[0088]

[0089] The bidirectional contrastive loss is a combination of the two contrastive losses:

[0090] L al = L Z + L θ

[0091] where τ ∈ R + is the temperature factor, A i := τ{i} is the index set of the contrastive samples, P i := {p ∈ A i : y p = y i} is the index set of the positive samples, and |P i | is the cardinality of P i .

[0092] L Z is the contrastive learning loss function for the training features. Using the feature Z i as the anchor, train its correlation with all positive sample labels. is the set of all label lists, τ ∈ R + is the temperature factor, A i := τ{i} is the index set of the contrastive samples. is the set of all negative sample labels, and L θ is the contrastive learning loss function for the training sample class labels. Using the label information θ i as the anchor, train its correlation with all positive sample features. Z p is the set of positive sample features, and Z a is the set of negative sample features.​

[0093] Joint training and prediction

[0094] This model also uses label information for contrast training and supervised prediction.

[0095] During training, to make full use of label information, a cross-entropy loss L improved by softmax is proposed ce , to maximize θ i of each input sample x i *Z i :

[0096]

[0097] The final loss function is the cross-entropy plus a contrast loss function adjusted by the hyperparameter λ. These three tasks simultaneously improve the quality of feature representation, label quality, classifier quality, and model stability.

[0098] The overall loss is:

[0099] L overall = λ1L cal +L ce + λ2L al

[0100] During classification, to make better use of the supervision information, a classification function incorporating label information is proposed.

[0101] For each sample, making full use of the label set generated by the label information, the similarity between the feature and each label is calculated, and it belongs to the class with the highest similarity value.

[0102]

[0103] 3. Experimental data settings

[0104] In virtual adversarial training, continuous pre-training is performed on the trained Bert model. In the present invention, the optimal perturbation size δ = 1*10 -5 , step size η = 1*10 -3 , variance σ of the initial perturbation = 1*10 -5 are obtained through experiments. These values maximize the effect of adversarial training while ensuring training efficiency.

[0105] To adapt the input format to the BERT series of pre-trained language models, for each input sentence, all the labels are listed as a token sequence and inserted before the input sentence, and they are separated by a special SEP token. A special CLS token is also inserted at the beginning of the input sequence, and a SEP token is appended at the end of the input sequence.

[0106] Both BERT and RoBERTa use positional embeddings to exploit the order of tokens in the input sequence, so if the class labels are listed in a fixed order, they will be associated with the positional embeddings. To mitigate the impact of the label order, the order of the labels is randomly changed before forming the input sequence during the training phase. During the testing phase, the label order remains unchanged.

[0107] Fine-tune the pre-trained BERT-base-uncased and RoBERTa-base models using the AdamW optimizer.

[0108] A training process consisting of 30 epochs is used, with a linear learning rate decay from an initial learning rate of 2×10^(-5) to a final learning rate of 10^(-5). To reduce the risk of overfitting, we adopted a weight decay parameter of 0.01 and a dropout probability of 0.1 on all layers.

[0109] In addition, we set the batch size of the dataset to 64. In the selection of hyperparameters, we used a grid search strategy to select the best λ value from {0.01, 0.05, 0.1} for weighting the contrastive loss and virtual adversarial loss in the loss function. Additionally, we empirically set the temperature coefficient τ to 0.1.

[0110] 4. Model Training and Testing

[0111] We compare VABCL with four supervised learning baselines on two models: a model trained using cross-entropy loss (CE), a model using both cross-entropy loss and standard supervised contrastive loss (CE+SCL), a model using cross-entropy loss and self-supervised contrastive loss (CE+CL), and a model using dual contrastive learning loss (DualCL), as shown in Table 1.

[0112] Table 1 Classification Comparison Results of Different Models

[0113]

[0114]

[0115] As can be seen from Table 1, VADCL in BERT and RoBERTa encoders achieved the best classification performance in all settings. Compared with BERT and RoBERTa that only use Dual Contrastive Learning (DualCL), the robustness and generalization performance of VADCL have been significantly improved. Compared with CE+CL, the average improvements of VADCL on BERT and RoBERTa are 0.15% and 0.28% respectively. In addition, the performance of VADCL on RoBERTa also exceeded the previously highest CE+CL, which can well reflect the role of virtual adversarial training in the model. At the same time, the performance of CE and CE+SCL also cannot exceed VADCL. This is because the CE method ignores relational samples and the CE+SCL method cannot directly learn the classifier for the classification task.

[0116] In addition, we found that virtual adversarial training and the dual contrast loss term contribute to the model achieving better performance on the three datasets. It shows that leveraging the relationships between samples helps the model learn better representations in contrastive learning.

[0117] Example Two

[0118] This example provides a supervised text classification system based on virtual adversarial and bidirectional contrastive learning, which is characterized by including:

[0119] A data acquisition module, which is used to acquire text sample data and corresponding label information;

[0120] A classification model training module, which is used to train a text classification model based on the text sample data and the corresponding label information to obtain a trained text classification model; wherein, the construction method of the text classification model includes:

[0121] Combining virtual adversarial training and contrastive learning, calculating the perturbation using the calculation method of the optimal perturbation in virtual adversarial training, combining the calculated perturbation with the original text data to generate adversarial text data for contrastive learning, and in the two contrastive learnings, through the bidirectional contrastive loss function, simultaneously learning the feature representation of the adversarial text data and its associated class label.

[0122] A text classification module, which is used to classify the text based on the trained text classification model to obtain a text classification result.

[0123] Example Three

[0124] This example provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the supervised text classification method based on virtual adversarial and bidirectional contrastive learning as described in Example One.

[0125] Example Four

[0126] This embodiment provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the steps in the supervised text classification method based on virtual confrontation and bidirectional contrast learning as described in Embodiment 1.

[0127] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program codes.

[0128] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0129] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0131] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0132] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A supervised text classification method based on virtual confrontation and bidirectional contrast learning, characterized in that Including the following steps: Obtain text sample data and corresponding label information; Train a text classification model based on the text sample data and the corresponding label information to obtain a trained text classification model; wherein, the construction method of the text classification model includes: Combine virtual adversarial training and contrastive learning, calculate the perturbation using the calculation method of the best perturbation in virtual adversarial training, combine the calculated perturbation with the text sample data to generate adversarial text data for contrastive learning, and in two contrastive learning processes, through a bidirectional contrastive loss function, simultaneously learn the feature representation of the adversarial text data and its associated class label; Classify the text based on the trained text classification model to obtain a text classification result; Wherein, the calculation method of the best perturbation in virtual adversarial training is: Input the sentence information and label information into two identical encoders, add a perturbation to the first encoder to generate a perturbed sample, output a clean sample from the second encoder, and use the perturbed sample as an anchor to train its relationship with the clean correct sample; Wherein, the bidirectional contrastive loss function includes a first contrastive loss function and a second contrastive loss function. In the first contrastive loss function, determine the true label POS of a sample, use the feature CLS of this sample as an anchor, the outputs with all other labels being POS are positive samples, and the labels of non-POS samples are negative samples; In the second contrastive loss function: determine the true label POS of a sample, use the output with this sample's label being POS as an anchor, then the CLS of the outputs with all other sample labels being POS are positive samples, and the CLS of non-POS samples are negative samples, wherein, CLS is the feature representation of this sample, and POS and NEG are the labels of this sample.

2. The supervised text classification method based on virtual confrontation and bidirectional contrast learning as claimed in claim 1, wherein, The bidirectional contrastive loss function simultaneously learning the feature representation of the adversarial text data and its associated class label includes: The first contrastive loss function is a contrastive learning loss function for training features, using the features as an anchor to train their correlation with all positive sample labels; the second contrastive loss function is a contrastive learning loss function for training sample class labels, using the label information as an anchor to train their correlation with all positive sample features.

3. The supervised text classification method based on virtual confrontation and bidirectional contrast learning as described in claim 1, wherein, The bidirectional contrastive loss function is: L al = L Z + L θ where L Z is the contrastive learning loss function of training features. Using the feature Z i as the anchor point, the correlation between it and all positive sample labels is trained. is the set of all label list sets, τ ∈ R + is the temperature factor, A i = τ{i} is the index set of contrast samples. is the set of all negative sample labels, L θ is the contrastive learning loss function of training sample class labels. Using the label information θ i as the anchor point, the correlation between it and all positive sample features is trained. Z p is the set of positive sample features, Z a is the set of negative sample features.

4. For the supervised text classification method based on virtual adversarial and bidirectional contrastive learning as described in claim 1, for each input sentence, insert a special CLS marker at the front of the input sequence and append a SEP marker at the end of the input sequence.

5. The supervised text classification method based on virtual confrontation and bidirectional contrast learning as claimed in claim 1, wherein During classification, calculate the similarity between the features extracted from the text and each label, and the category of the label with the highest similarity is the category of this text.

6. A supervised text classification system based on virtual confrontation and bidirectional contrast learning, characterized in that, Adopt the supervised text classification method based on virtual adversarial and bidirectional contrastive learning as described in any one of claims 1-5, including: A data acquisition module, which is used to obtain text sample data and corresponding label information; A classification model training module, which is used to train a text classification model based on the text sample data and the corresponding label information to obtain a trained text classification model; wherein, the construction method of the text classification model includes: Combining virtual adversarial training and contrastive learning, calculating the perturbation using the calculation method of the optimal perturbation in virtual adversarial training, combining the calculated perturbation with the text sample data to generate adversarial text data for contrastive learning, and in two contrastive learning processes, simultaneously learning the feature representation of the adversarial text data and its associated class label through a bidirectional contrastive loss function; A text classification module, which is used to classify the text based on the trained text classification model to obtain a text classification result.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps in the supervised text classification method based on virtual adversarial and bidirectional contrastive learning as described in any one of claims 1-5.

8. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the supervised text classification method based on virtual adversarial and bidirectional contrastive learning as described in any one of claims 1-5.

Images