Hardened Deep Neural Networks via Iterative Adversarial Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face challenges in reliably and efficiently classifying images, particularly when faced with adversarial images or malware that can evade detection by altering minor portions of the code.
Innovation Solution
A DNN training system that iteratively trains DNNs using images misclassified by previous iterations, generating adversarial images through analytical attacks, and incorporating these images into the training set to enhance classification accuracy and robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a DNN is trained with traditional methods on standard training sets, then it achieves basic classification capability, but it becomes vulnerable to adversarial images and malware that can evade detection by altering minor portions
Solution Approach 1:
The patent applies preliminary anti-action by generating adversarial images through analytical attacks before final deployment. These adversarial examples are created by intentionally modifying input images to cause misclassification, then incorporating them into the training set to pre-harden the DNN against such attacks before real-world deployment
Solution Approach 2:
The patent converts the harmful adversarial images into beneficial training data. By using analytical attacks to generate adversarial examples and then incorporating them into the training set, the system transforms previously harmful inputs that caused misclassification into valuable training samples that improve the DNN's robustness and classification accuracy
2Reliability
If a DNN is hardened through iterative training with adversarial images, then classification accuracy for adversarial images improves, but training time and computational resources increase
Solution Approach 1:
The patent performs preliminary action by generating adversarial images and incorporating them into the training set before final model deployment. This iterative process of generating adversarial examples and retraining is performed in advance to harden the DNN, reducing the need for post-deployment adjustments and potentially reducing inference time later
Solution Approach 2:
The patent implements periodic action through iterative training cycles. The DNN is trained, evaluated against adversarial images, retrained with incorporated adversarial examples, and this cycle repeats for a predetermined number of iterations or until convergence, systematically improving robustness over multiple periodic training phases
Data Source
AI summary
Various embodiments are generally directed to techniques for training deep neural networks, such as with an iterative approach, for instance. Some embodiments are particularly directed to a deep neural network (DNN) training system that generates a hardened DNN by iteratively training DNNs with images that were misclassified by previous iterations of the DNN. One or more embodiments, for example, may include logic to generate an adversarial image that is misclassified by a first DNN that was previously trained with a set of sample images. In some embodiments, the logic may determine a second training set that includes the adversarial image that was misclassified by the first DNN and the first training set of one or more sample images. The second training set may be used to train a second DNN. In various embodiments, the above process may be repeated for a predetermined number of iterations to produce a hardened DNN.