Neural Network Learning System Using Intermediate Concept Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep neural networks (DNNs) are susceptible to adversarial attacks and require large amounts of annotated training data, making them vulnerable to noisy inputs and domain changes.
Innovation Solution
The system employs a robust neural network learning system that incorporates intermediate concept constraints and inferencing during training, using constraints such as data constraints, feature constraints, and intermediate concept constraints to enhance resilience against adversarial attacks and improve generalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional DNNs are trained with large amounts of annotated training data, then classification accuracy is improved, but the system becomes vulnerable to adversarial attacks and noisy inputs
Solution Approach 1:
The patent introduces intermediate concepts as mediator layers between the input data and final classification. These intermediate concepts act as a buffer that constrains the feature representations to lie within predefined concept spaces, preventing adversarial inputs from causing arbitrary feature transformations. The intermediate concepts serve as a protective intermediary that maintains classification accuracy while filtering out adversarial perturbations.
Solution Approach 2:
The patent transforms the training objective by changing the parameters being optimized. Instead of only optimizing for classification accuracy on training data, the system optimizes for consistency with intermediate concept constraints. This parameter change in the loss function allows the network to learn features that are both accurate and robust to adversarial attacks by enforcing structural constraints on the feature representations.
2Measurement precision
If DNNs are trained with extensive annotated training data, then model performance is improved, but the requirement for large amounts of annotated data increases
Solution Approach 1:
The patent performs preliminary action by predefining intermediate concept spaces and constraints before training the neural network. These intermediate concepts are established a priori based on domain knowledge or preliminary analysis, allowing the main training process to focus on learning within these constrained spaces rather than discovering all features from scratch. This preliminary structuring reduces the amount of training data needed.
Solution Approach 2:
The introduction of intermediate concept constraints changes the parameter space in which the network learns. By constraining features to lie within predefined concept spaces, the effective dimensionality and complexity of the learning problem are reduced, allowing high performance to be achieved with less training data.
3Productivity
If conventional DNNs process noisy input data, then the network attempts to make predictions, but the predictions become inaccurate due to adversarial attacks
Solution Approach 1:
The intermediate concepts serve as a protective intermediary layer that filters noisy and adversarial inputs before they reach the final classification layer. By constraining the feature representations to lie within predefined concept spaces, the system maintains prediction capability while preventing adversarial inputs from causing inaccurate predictions. The intermediary concept space acts as a buffer that preserves prediction functionality while filtering out harmful perturbations.
Data Source
AI summary
A system comprises a computer including a processor and a memory. The memory includes instructions such that the processor is programmed to: receive intermediate concept constraints at a neural network and train the neural network with training data, training labels, and the at least one of the data constraint, the feature constraint, or the intermediate concept constraint.