Text Classification Model Adversarial Training Robustness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional text classification models are not robust and can make errors when small perturbations are added to the input sentence.
Innovation Solution
A method for training a text classification model that involves obtaining a training sample, determining its semantic representation, generating an adversarial sample with perturbation information, calculating classification and contrastive losses, and optimizing the model parameters based on these losses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional text classification models are used, then the model structure is simple and training is fast, but the model lacks robustness and makes errors when small perturbations are added to the input sentence
Solution Approach 1:
The patent applies preliminary action by generating adversarial samples before actual classification tasks. The model is pre-trained with perturbed samples that have semantic modifications, allowing it to learn robust features in advance. This prepares the model to handle perturbations during deployment without requiring complex architectural changes.
Solution Approach 2:
The patent employs parameter changes by modifying the input text parameters through semantic perturbations (synonym replacement, word insertion, deletion). During training, the model learns to maintain classification accuracy despite these parameter changes, effectively improving robustness through parameter variation rather than structural complexity.
2Reliability
If adversarial samples with perturbation information are generated and used for training, then the robustness of the model is improved, but the training process becomes more complex and time-consuming
Solution Approach 1:
The patent applies partial action by generating adversarial samples with controlled perturbation levels. Instead of using extreme or excessive perturbations, the method uses semantic modifications that are realistic and bounded, achieving robustness improvement without requiring exhaustive training on all possible perturbations. This reduces training time while maintaining effectiveness.
Solution Approach 2:
The patent uses copying by creating adversarial versions of existing training samples rather than generating entirely new data. The perturbed samples are copies of original training data with semantic modifications, allowing the model to learn from familiar patterns while improving robustness. This approach is more time-efficient than creating new training data from scratch.
3Reliability
If the model is trained to correctly classify both original and adversarial samples, then the classification accuracy under perturbation is improved, but the training complexity and computational resources increase
Solution Approach 1:
The patent applies merging by combining the training of original samples and adversarial samples into a unified training process. The loss function integrates both clean and perturbed sample classifications, allowing the model to learn robust features efficiently. This combined approach reduces computational overhead compared to separate training processes while improving accuracy under perturbation.
Data Source
AI summary
This disclosure relates to a method and an apparatus for training a text classification model. The method may include determining a semantic representation of the training sample using the text classification model and determining a predicted classification result of the training sample based on the semantic representation. The method may further include generating an adversarial sample corresponding to the training sample based on the training sample and perturbation information and determining a semantic representation of the adversarial sample corresponding to the training sample using the text classification model. The method may further include determining a classification loss of the text classification model based on the predicted classification result, determining a contrastive loss of the text classification model based on the semantic representation and the semantic representation of the adversarial sample, and training the text classification model based on the classification loss and the contrastive loss.