Blind Learning Neural Network Split for Sequential Data Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing approaches to training neural networks, such as federated learning and split learning, face limitations in maintaining data privacy and efficiency, particularly in handling sequential models like RNN, LSTM, and GRU, and are not well-suited for predicting the remaining useful life of systems using convolutional neural networks.
Innovation Solution
The introduction of a blind learning approach that splits neural networks into client-side and server-side portions, allowing parallel and independent processing across clients, and enables the training of sequential models by reducing dimensionality of input data, while maintaining data privacy through encryption and aggregation techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If federated learning or split learning approaches are used to train neural networks, then data privacy is maintained, but training efficiency and support for sequential models are limited
Solution Approach 1:
The neural network is segmented into multiple independent portions distributed across different clients. Each client trains local portions independently on their own data, avoiding centralized data collection while enabling parallel training. This segmentation resolves the contradiction by maintaining privacy (data stays local) while improving efficiency (parallel processing across clients).
Solution Approach 2:
The patent introduces a new dimension of model distribution by splitting the neural network across multiple spatial locations (clients) rather than just centralizing it. This dimensional change enables simultaneous local training on encrypted data while maintaining coordination through secure aggregation, thus achieving both privacy and efficiency.
2Reliability
If existing federated or split learning approaches are used, then data privacy is preserved, but support for sequential models like RNN, LSTM, and GRU is limited
Solution Approach 1:
The patent creates a universal blind learning framework that can accommodate multiple model types including sequential models (RNN, LSTM, GRU), convolutional networks, and fully-connected networks. By designing portioned models with standardized interfaces and using encryption techniques that work with any model architecture, the system achieves versatility while maintaining privacy.
Solution Approach 2:
The patent changes the parameter of model architecture flexibility by allowing different portions of the network to have different structures suitable for different data types. Sequential model portions can be assigned to clients with sequential data, while other portions handle different modalities, all within the same private training framework.
3Productivity
If centralized training approaches are used, then training efficiency is improved, but data privacy and security are compromised
Solution Approach 1:
The patent introduces cryptographic intermediaries (encryption schemes, secure aggregation protocols) that mediate between the need for efficient centralized coordination and the requirement for data privacy. These intermediaries enable the system to achieve centralized training efficiency without directly exposing raw data, as all communications are encrypted and aggregated securely.
4Reliability
If neural networks are split across clients for private training, then data privacy is maintained, but device complexity and coordination overhead increase
Solution Approach 1:
By segmenting the model into portions and assigning them to different clients, the patent reduces the computational burden and memory requirements on each individual device. Each client only needs to store and process their assigned portion, simplifying local device requirements while the overall system achieves privacy through distribution.
Data Source
AI summary
A system and method are disclosed related to building a predictive model from sequential data using convolutional neural networks such as predicting the remaining useful life of a system. An example method includes organizing training data into a two-dimensional format, normalizing the training data to yield normalized training data, simulating a sequence model using a one-dimensional convolutional neural network, collecting feature maps that result from previous layers in the one-dimensional convolutional neural network into a single layer, inputting an output from the single layer into a fully connected network and predicting, based on the fully connected network operating on the output of the single layer, a target value associated with the training data.