Adversarial Training Data Augmentation for NLP Robustness

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing natural language processing systems face challenges in efficiently identifying and understanding dynamic structures and changes in natural language, leading to difficulties in processing content responses effectively.

Innovation Solution

An AI platform with a classifier using adversarial training sets to predict classification labels, which identifies intent and corpus corresponding to input data, and produces response data reflecting the input, by leveraging synthetic data created through perturbation-based data augmentation and adversarial training.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If adversarial training sets are used to improve robustness against dynamic structures, then reliability is improved, but device complexity increases

Engineering Contradiction:

Improverobustness to dynamic structuresVSAvoidtraining data structure

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system performs preliminary actions by generating synthetic adversarial training data before the classification model is deployed. Adversarial examples are created by applying perturbations to original training data, and the model is pre-trained on both original and adversarial examples to build robustness in advance, rather than dealing with dynamic structures during inference

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system changes parameters by introducing controlled perturbations to the training data. Adversarial examples are generated by modifying input parameters (e.g., adding small noise, changing word order, synonym replacement) to create variations that test and improve the model's robustness to dynamic structural changes

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If perturbation-based data augmentation is applied to handle dynamic changes, then adaptability is improved, but manufacturing precision deteriorates

Engineering Contradiction:

Improvehandling dynamic changesVSAvoidclassification accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The system converts potentially harmful perturbations (which could degrade classification accuracy) into beneficial training signals. By deliberately introducing adversarial perturbations during training, the model learns to recognize and correctly classify perturbed inputs, turning the harm of accuracy degradation into the benefit of improved robustness and adaptability

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Solution Approach 2:

The system implements feedback by using the classification model's performance on adversarial examples to guide further training. The model is trained on original data, evaluated on adversarial examples, and retrained with newly generated adversarial data that targets its weaknesses, creating a feedback loop that continuously improves both accuracy and adaptability

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS11189269B2Adversarial training data augmentation for generating related responses

Publication Date: 2021.11.30 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

An intelligent computer platform to introduce adversarial training to natural language processing (NLP). An initial training set is modified with synthetic training data to create an adversarial training set. The modification includes use of natural language understanding (NLU) to parse the initial training set into components and identify component categories. As input is presented, a classifier evaluates the input and leverages the adversarial training set to identify the intent of the input. An identified classification model generates accurate and reflective response data based on the received input.