Neural Network Activation Function Obfuscation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks are vulnerable to adversarial attacks that aim to reveal proprietary information such as weights, biases, and activation functions, which can compromise their security and integrity.
Innovation Solution
The implementation of obfuscation techniques, including expanding weights and biases with dummy values, masking using linear or reversible non-linear transformations, and obfuscating activation functions, as well as introducing dummy operations and nodes to obscure the neural network's architecture and computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neural networks implement standard activation functions and weight-bias structures, then computational efficiency and model performance are optimized, but security against adversarial attacks deteriorates
Solution Approach 1:
The patent introduces obfuscation functions as intermediary components between the input and the activation function. These obfuscation functions transform the input data in a reversible manner, creating an intermediate representation that masks the actual activation function being applied. This intermediary layer prevents attackers from directly observing or inferring the activation function while maintaining the correct computational outcome.
Solution Approach 2:
The patent applies parameter changes by transforming the activation function into a composite function that combines multiple mathematical operations with different parameters. Instead of using a single standard activation function, the system uses a family of activation functions with varying parameters, making it difficult for attackers to identify the specific function being used while maintaining functional equivalence for legitimate operations.
2Reliability
If neural networks use standard weight and bias representations, then training efficiency is maximized, but protection against weight extraction attacks deteriorates
Solution Approach 1:
The patent creates a copy of the weight-bias structure by introducing additional dummy weights and biases that mirror the actual weight-bias parameters. These dummy parameters are constructed to have the same dimensional structure and computational role as the real parameters, making them indistinguishable to attackers through statistical analysis or gradient-based extraction methods, while remaining functionally equivalent for training purposes.
Solution Approach 2:
The patent segments the weight-bias representation into multiple independent components, separating the actual functional parameters from dummy obfuscation parameters. This segmentation allows the system to maintain the necessary computational functionality through the actual parameters while distributing the obfuscation burden across multiple dummy parameters, thereby protecting against extraction attacks without significantly impacting training efficiency.
3Reliability
If neural networks implement comprehensive obfuscation of weights, biases, and activation functions, then security against adversarial attacks is enhanced, but computational performance deteriorates
Solution Approach 1:
The patent applies partial obfuscation by selectively obfuscating only the critical components (activation functions and key weight-bias parameters) while leaving other parts of the network unchanged. This partial action provides sufficient security protection against adversarial attacks while minimizing the computational overhead and performance impact that would result from comprehensive obfuscation of all network elements.
Data Source
AI summary
Aspects of the present disclosure involve implementations that may be used to protect neural network models against adversarial attacks by obfuscating neural network operations and architecture. Obfuscation techniques include obfuscating weights and biases of neural network nodes, obfuscating activation functions used by neural networks, as well as obfuscating neural network architecture by introducing dummy operations, dummy nodes, and dummy layers into the neural networks.