Failure Image Variety Simulation for Robust Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating diverse images to improve deep learning neural network robustness are computationally expensive and require extensive data acquisition, limiting their practical application.
Innovation Solution
A novel loss function-based approach trains a second deep learning neural network to learn the visual variety of a failure image using dummy images, without the need for specially-curated paired or unpaired images, and applies this network to convert training images, enhancing their diversity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing techniques are used to generate diverse images for improving neural network robustness, then the neural network's robustness is improved, but computational costs and data acquisition requirements increase excessively
Solution Approach 1:
The patent creates synthetic copies of failure images by training a second neural network to generate images with similar visual characteristics (color histograms, gradients, textures) to the failure image. These synthetic copies serve as augmented training data, replacing the need for expensive manual data collection and processing while improving neural network robustness through exposure to varied visual presentations of failure cases
Solution Approach 2:
The system uses the failure image itself as the source material for generating augmented training data. The second neural network learns from the failure image's visual properties and automatically generates diverse synthetic variations, eliminating the need for external data sources or manual intervention. The failure case serves its own purpose of teaching the system how to generate protective training examples
2Reliability
If existing techniques are used to generate diverse images for improving neural network robustness, then the neural network's robustness is improved, but data acquisition requirements increase excessively
Solution Approach 1:
The patent creates synthetic copies of failure images by training a second neural network to generate images with similar visual characteristics (color histograms, gradients, textures) to the failure image. These synthetic copies serve as augmented training data, replacing the need for expensive manual data collection and processing while improving neural network robustness through exposure to varied visual presentations of failure cases
Solution Approach 2:
The system uses the failure image itself as the source material for generating augmented training data. The second neural network learns from the failure image's visual properties and automatically generates diverse synthetic variations, eliminating the need for external data sources or manual intervention. The failure case serves its own purpose of teaching the system how to generate protective training examples
3Adaptability or versatility
If a second deep learning neural network is trained to learn visual variety of failure images using a loss function with multiple terms, then image diversity is improved, but device complexity increases
Solution Approach 1:
The loss function is segmented into distinct terms, each responsible for evaluating a specific aspect of image similarity (color histogram matching, gradient magnitude comparison, texture analysis). This modular approach to the loss function allows the complex task of multi-dimensional image comparison to be broken down into manageable, independent computational components that can be implemented and tuned separately
Solution Approach 2:
The patent introduces a second neural network as an intermediary between the failure image and the training process. This intermediary network learns to generate images that match the visual characteristics of failure images, serving as a bridge that translates the complex requirement of 'visual variety' into concrete image transformations. The intermediary network simplifies the overall system by encapsulating the complexity of multi-term loss function evaluation within its training process
Data Source
AI summary
In various embodiments, a system can: access a failure image on which a first model has inaccurately performed an inferencing task; train, on a set of dummy images, a second model to learn a visual variety of the failure image, based on a loss function having a first term and a second term, the first term quantifying visual content dissimilarities between the set of dummy images and outputs predicted during training by the second model, and the second term quantifying, at a plurality of different image scales, visual variety dissimilarities between the failure image and the outputs predicted during training by the second model; and execute the second model on each of a set of training images on which the first model was trained, thereby yielding a set of first converted training images that exhibit the visual variety of the failure image.