Adversarial Neural Network Training for Robust Text Comprehension
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine reading and comprehension systems are limited by strict supervised training methods, which do not adequately prepare them for real-world noise and variability in text data.
Innovation Solution
The proposed solution involves training a machine reading network using an adversarial method, where a second neural network obfuscates words in text corpuses to challenge the first neural network's ability to answer questions, thereby enhancing its robustness and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If strict supervised training methods are used, then training process is simple and controlled, but the system lacks robustness to noise and variability in real-world text data
Solution Approach 1:
The patent introduces an adversarial training mechanism where a second neural network deliberately obfuscates words in text corpuses to create challenging training examples. This converts the harmful effect of noise and variability into a beneficial training process, forcing the first neural network to learn robust representations that can handle real-world text data despite obfuscation. The adversarial approach transforms potential training failures into opportunities for improving system robustness.
Solution Approach 2:
The system performs preliminary obfuscation of text data before it reaches the first neural network for processing. By pre-corrupting the text corpuses with the second neural network, the system prepares challenging training examples in advance that simulate real-world noise and variability conditions, enabling the first network to develop resilience before encountering such data in production.
2Reliability
If adversarial training with obfuscation is applied, then robustness to noise is improved, but training time and computational resources increase
Solution Approach 1:
The patent implements a feedback loop where the second neural network evaluates the performance of the first neural network on obfuscated text and adjusts the obfuscation strategy accordingly. The system monitors answering performance metrics and uses this feedback to dynamically control the adversarial training process, allowing efficient use of computational resources by focusing on the most impactful training examples rather than uniformly processing all data.
Solution Approach 2:
The system applies obfuscation selectively rather than uniformly to all text data. The second neural network identifies and obfuscates only the most critical words or passages that will provide maximum training benefit, rather than corrupting entire texts or all words. This partial action approach reduces unnecessary computational overhead while maintaining the robustness benefits of adversarial training.
Data Source
AI summary
A method of using a first neural network includes: by the first neural network, receiving a text; by the first neural network, receiving a question concerning the text; and by the first neural network, determining an answer to the question using the text, where the first neural network is trained to answer the question about the text adversarially by a second neural network that is trained to maximize a likelihood of failure of the first neural network to correctly answer questions.