Sanitizing Neural Networks by Removing Malicious Parameters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network models can be compromised by malicious code embedded within their parameters, which can go undetected and cause system infections when downloaded and executed, evading traditional malware protection suites.
Innovation Solution
A security platform that sanitizes neural network models by retraining them using a process involving input nodes, error value determination, and parameter updates based on loss functions, such as mean squared error or cross-entropy, to identify and neutralize malicious code without affecting the model's functionality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional malware protection suites are used to detect malicious code, then detection capability is limited, but neural network models can still be compromised by embedded malicious code
Solution Approach 1:
The patent applies preliminary action by retraining the neural network model before deployment to sanitize its parameters. The system performs model retraining with monitoring of parameter changes, identifying and removing malicious code embedded in the model parameters before the model is used in production, thus preventing potential malware execution while maintaining model functionality
Solution Approach 2:
The patent uses an intermediary approach by introducing a security platform that acts as a mediator between model download and model deployment. This platform performs retraining and sanitization operations, serving as an intermediate processing step that cleans the model parameters of malicious code while preserving the legitimate model functionality
2Reliability
If model parameters are extensively modified to remove malicious code, then security is improved, but model functionality may be affected
Solution Approach 1:
The patent applies feedback by continuously monitoring parameter changes during the retraining process. The system tracks the quantity of updated parameters and compares it against thresholds, using this feedback to determine when to stop the retraining process. This ensures that malicious code is removed while preventing excessive modifications that could degrade model performance
Solution Approach 2:
The patent uses parameter changes as a controlled mechanism for sanitization. By monitoring the quantity and magnitude of parameter updates during retraining, the system identifies malicious code embeddings and removes them through targeted parameter modifications, balancing security requirements with model functionality preservation
Data Source
AI summary
Systems, devices, and methods for protecting a user computer devices/network from malicious code embedded in a neural network is described. A security platform may selectively modify a downloaded neural network model and/or architecture to remove neural network parameters that may be used to reconstruct the malicious code at an end user of the neural network model. For example, the security platform may remove specific branches of the neural network and/or set specific parameters of the neural network model to zero, such that the malicious code may not be reconstructed at an end-user device.