Deep Neural Network Model Fusing for Adversarial Robustness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network models face challenges in enhancing performance and robustness against adversarial attacks, particularly in resource-constrained environments, where ensemble schemes require significant resources and memory to combine multiple models.
Innovation Solution
A method involving a single deep neural network (DNN) model that changes its connection weights using noise values from a noise model, generating multiple output data sets which are then combined to produce a final result, thereby reducing resource usage while enhancing accuracy and robustness against adversarial attacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple DNN models are used in an ensemble scheme to enhance performance and robustness, then accuracy and robustness against adversarial attacks are improved, but resource consumption and memory requirements increase significantly
Solution Approach 1:
The patent merges multiple DNN models into a single unified model that incorporates parameters from multiple source models. Instead of maintaining separate model instances in memory, the system combines their parameters (weights and biases) into one model structure, achieving ensemble-like robustness while reducing memory footprint to that of a single model.
Solution Approach 2:
The single DNN model is designed to perform multiple functions by integrating parameters from different source models trained on different datasets or with different architectures. This universal model can handle various classification tasks that would otherwise require multiple specialized models, reducing overall resource consumption.
2Productivity
If multiple DNN models are deployed to improve classification accuracy, then performance is enhanced, but computational resources and processing power increase
Solution Approach 1:
The patent combines the computational capabilities of multiple models into a single model by merging their parameters. The unified model processes inputs through a single forward pass rather than requiring multiple separate inferences, reducing computational overhead while maintaining the accuracy benefits of ensemble approaches.
Solution Approach 2:
The system changes the parameters of a single DNN model by incorporating weights and biases from multiple source models. This parameter-level transformation allows the model to leverage diverse training experiences without requiring multiple model instances, thereby improving accuracy while controlling computational resource usage.
3Reliability
If ensemble schemes are implemented to increase robustness against adversarial attacks, then reliability is improved, but device complexity and implementation difficulty increase
Solution Approach 1:
The patent simplifies the ensemble implementation by merging multiple model parameters into a single model structure. This eliminates the complexity of managing multiple model instances, their separate loading, and coordination during inference. The system only needs to manage one model file and one inference pipeline, significantly reducing implementation complexity.
Solution Approach 2:
The patent extracts the essential robustness characteristics from multiple ensemble models and consolidates them into a single model's parameters. By separating the core functionality (parameter combinations that provide robustness) from the overhead of managing multiple models, the system achieves reliability without the associated complexity.
Data Source
AI summary
A processor-implemented method including implementing a deep neural network (DNN) model using input data, generating, by implementing the DNN model, first output data from the DNN model, changing the DNN model, generating, by implementing the changed DNN model using the input data, second output data of the changed DNN model, and determining result data by combining the first output data and the second output data.