Protected Neural Networks With Partial Weight Encryption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Completely encrypted neural networks are computationally intensive to decrypt, leading to prolonged startup times in systems, which can exceed system startup requirements and impact performance, especially in resource-constrained environments like autonomous vehicles.
Innovation Solution
Implementing a machine learning/AI system where only a portion of the neural network weights and biases are encrypted, using techniques such as partial encryption, keyed permutations, exclusive OR functions, and stream ciphers to protect against reverse engineering, while minimizing decryption time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entirety of the neural network is encrypted to prevent adversaries from gaining access to the weights and biases, then security against reverse engineering is improved, but decryption time increases and startup performance deteriorates
Solution Approach 1:
The neural network is divided into multiple layers, and only a portion of the weights (specifically those in the last half of the layers) are encrypted while other weights remain unencrypted. This segmentation allows the system to maintain security for critical parameters while avoiding the need to decrypt the entire network, thereby reducing decryption time and improving startup performance.
Solution Approach 2:
Different encryption strategies are applied to different parts of the neural network based on their importance. The patent encrypts weights in later layers that are more consequential to the network's function, while leaving earlier layer weights unencrypted. This local quality approach ensures that security is focused where it is most needed, balancing security requirements with computational efficiency.
2Reliability
If the entirety of the neural network is encrypted, then protection against reverse engineering is improved, but computational resources and hardware complexity increase
Solution Approach 1:
By encrypting only a segment of the neural network weights rather than the entire network, the patent reduces the computational burden and hardware requirements for decryption. The segmented approach allows standard hardware to handle the limited decryption needed, avoiding the requirement for specialized hardware accelerators.
Solution Approach 2:
The patent applies partial encryption to only the most critical portion of the neural network weights (those in the last half of layers). This partial action provides sufficient protection against reverse engineering for the most important parameters without requiring full encryption of all weights, thereby reducing hardware complexity and computational resource requirements.
Data Source
AI summary
A machine learning/artificial intelligence (ML/AI) system includes one or more controllers storing one or more neural networks in memory. The one or more neural networks include a plurality of layers including an input layer, one or more hidden layers, and an output layer, and one or more nodes provided for each of the plurality of layers of the one or more neural networks. Each of the plurality of layers are connected to a subsequent layer of the one or more neural networks by a connection. The connection connects a first node of an earlier layer with a second node of a later layer. The one or more neural networks also include plurality of weights, where each weight is associated with a connection and only a portion of the plurality of weights of the one or more neural networks are encrypted or protected.