Distributed Neural Network Training with Local Subsection Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks face challenges in training with private data, as sharing sensitive information is often restricted, hindering performance improvements.
Innovation Solution
A distributed training approach where a central computing system maintains a partially trained neural network model and partners maintain subsections, training locally without sharing data, with parameters combined and updated to form a complete model for anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If training data is shared across multiple systems to improve neural network performance, then model accuracy improves, but data security and privacy are compromised
Solution Approach 1:
The neural network model is divided into multiple subsections, with each partner system training and maintaining a specific subsection locally. This segmentation allows each system to contribute to the overall model without sharing sensitive training data, thereby improving model accuracy while preserving data security and privacy.
2Object-affected harmful factors
If each partner system trains independently with private data, then data security is maintained, but model performance is limited due to lack of data sharing
Solution Approach 1:
The independently trained neural network subsections from multiple partner systems are merged into a complete model by the central computing system. This merging combines the benefits of private data training with collaborative model improvement, maintaining data security while enhancing overall model performance through aggregation of specialized subsections.
3Measurement precision
If a central system collects all training data for centralized training, then model performance improves, but data privacy and security requirements cannot be met
Solution Approach 1:
The training data is extracted from the central system and distributed to partner systems for local training of specific model subsections. This extraction eliminates the need for centralized data collection, allowing high-performance training while maintaining data privacy and security through distributed processing.
4Device complexity
If neural network subsections are combined in series, then model depth and complexity increase, but training coordination and convergence become more difficult
Solution Approach 1:
The central computing system coordinates the training of series-combined subsections by collecting parameters from each partner system and distributing updated model information back to all participants. This feedback mechanism ensures proper convergence and coordination of the distributed training process while maintaining the benefits of increased model depth and complexity.
Data Source
AI summary
Techniques are disclosed relating to training a neural network using private training data. In some embodiments, a central computing system is configured to maintain an at least partially trained neural network and information that specifies data formats for inputs to the model and outputs from the model. In some embodiments, partner computing systems maintain subsections of the neural network model and may train them using data that is not shared with other partner computing systems or the central computing system. Parameters resulting from the training may be transmitted to the central computing system. In some embodiments, the central computing system processes the parameters to generate the updated complete version of the neural network model and transmits parameters from the updated complete version of the model to the partner computing systems. The partner computing systems may use the updated complete model to detect anomalies in input data.