Privacy-Preserving Neural Network Using Cosine Activation and Hadamard Matrices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional secure multi-party computation (MPC) environments for neural networks are resource-intensive and leak intermediate results, undermining data privacy during training and inference, as they require multiple communication rounds and dense matrices, leading to inefficient computational and network resource usage.
Innovation Solution
Implementing a cosine activation function in a two-party computing setup using Hadamard-Diagonal matrices, which reduces the number of secure multiplications and communication rounds, ensuring privacy preservation by using structured matrices that mimic the performance of existing architectures while minimizing bandwidth consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional secure multi-party computation (MPC) environments are used for neural networks, then data privacy is preserved through secret sharing, but computational and network resource usage becomes excessive due to multiple communication rounds and dense matrices
Solution Approach 1:
The patent changes the parameter of matrix structure from dense to structured (Hadamard-diagonal), which fundamentally alters the computational characteristics. This parameter change enables the system to maintain security through secret sharing while dramatically reducing the number of communication rounds and computational operations required, thus resolving the contradiction between privacy preservation and computational efficiency
Solution Approach 2:
The patent segments the weight matrix into structured components (Hadamard-diagonal form) that can be processed more efficiently. By segmenting the computation into fewer, more structured operations, the system reduces the number of communication rounds between parties while maintaining the security guarantees of MPC, thereby improving productivity without sacrificing reliability
2Adaptability or versatility
If conventional MPC environments use dense matrices for neural network computations, then complete model functionality is maintained, but network bandwidth consumption increases due to multiple communication rounds
Solution Approach 1:
The patent applies parameter changes by transforming the matrix representation from dense to structured (Hadamard-diagonal). This change reduces the communication overhead significantly while preserving the essential functionality of the neural network model, thus reducing network bandwidth consumption without losing adaptability
Solution Approach 2:
The patent extracts and removes unnecessary computational complexity from the dense matrix operations. By taking out the redundant elements of dense matrix multiplication and replacing them with structured operations, the system maintains model functionality while reducing network bandwidth consumption associated with multiple communication rounds
3Reliability
If conventional MPC environments perform secure multiplications using secret shares, then data privacy is maintained, but the number of secure multiplication operations becomes computationally expensive
Solution Approach 1:
The patent changes the computational parameter from general dense matrix multiplication to structured Hadamard-diagonal multiplication. This parameter change reduces the number of secure multiplication operations required while maintaining privacy preservation through secret sharing, thus reducing device complexity
Solution Approach 2:
The patent uses properties of Hadamard matrices to create a structured computation pattern that copies and reuses computational intermediates more efficiently. This approach reduces the total number of secure multiplication operations needed while maintaining the security guarantees, thereby reducing computational complexity
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for implementing privacy-preserving machine learning models (e.g., neural networks) in secure multi-part computing environments. Methods can include computing an output of a particular layer of a neural network deployed in a two computing system environment using a cosine activation function.