Split Layer Data Leakage Quantification in Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing approaches to training neural networks, such as federated learning and split learning, face challenges in maintaining data privacy and efficiency, particularly due to data leakage issues in decentralized datasets.
Innovation Solution
The proposed solution introduces a statistical metric to quantify data leakage from a split layer in neural network training paradigms, specifically using a blind learning approach that enables parallel and independent client-side processing, and incorporates sequential models like RNN and LSTM.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If split learning is used to train neural networks on decentralized datasets, then training efficiency and data privacy are improved, but data leakage from the split layer occurs
Solution Approach 1:
The patent introduces an intermediary metric (statistical distance measurement) that quantifies data leakage from the split layer. This intermediary allows the system to measure and control the harmful effect without eliminating the beneficial split learning approach, enabling informed decisions about model architecture and training parameters.
Solution Approach 2:
The patent implements a feedback mechanism by calculating statistical distance metrics between original and reconstructed data. This feedback loop allows the system to evaluate data leakage and adjust the split layer configuration, network architecture, or training parameters to minimize leakage while maintaining training efficiency.
2Adaptability or versatility
If sequential models like RNN and LSTM are incorporated into split learning, then model versatility is improved, but implementation complexity increases
Solution Approach 1:
The patent creates a universal framework that can handle both traditional feedforward networks and sequential models (RNN, LSTM) using the same split learning architecture. The quantification metric and training methodology are designed to be model-agnostic, allowing the system to process diverse data types and model architectures through a unified approach.
Data Source
AI summary
A system and method are disclosed for providing an artificial intelligence platform. An example method includes examining part of a global neural network to locate a split layer in the global neural network, creating an equivalent model to the part of the global neural network of a same size but having opposite operations, generating smashed data based on an operation on input data by the part of the global neural network, training the equivalent model by inputting the smashed data to generate a second a mirrored copy of the input data, quantifying a distance between the input data and the second generated set of mirrored data to yield a value and, when the value is below a threshold, determining that a current location of the split layer in the global neural network is safe for a training process.