Distributed Neural Network Training with Privacy Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed training of artificial neural networks faces challenges with data privacy concerns and network performance issues, particularly when using multiple processors or distributed computing, as sensitive data needs to be protected while maintaining network efficiency.
Innovation Solution
A system where multiple computing devices host local versions of an artificial neural network (ANN), allowing local training without sharing user data, with changes and results transmitted to combine into a master version, ensuring data privacy while updating the network, and allowing the master version to be downloaded for improved local training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If distributed computing with multiple processors is used to train neural networks, then training effectiveness and network accuracy are improved, but data privacy concerns arise and network performance is limited
Solution Approach 1:
The patent segments the neural network training process into local training on individual devices and centralized aggregation of model updates. Each device trains its local copy of the neural network independently on its own data, and only model parameter changes (not raw data) are transmitted to the central server for aggregation. This segmentation allows distributed training to improve accuracy while protecting data privacy, as the sensitive data never leaves the local device.
2Productivity
If distributed computing with multiple processors is used to train neural networks, then training effectiveness is improved, but network performance is limited
Solution Approach 1:
The patent extracts the computationally intensive training operations from the centralized cloud environment and places them on local devices. By taking out the heavy lifting of forward propagation and backpropagation to individual devices that can perform these operations locally, the system avoids network bandwidth bottlenecks and latency issues associated with centralized distributed training, thereby improving overall network performance while maintaining training effectiveness.
3Object-affected harmful factors
If local training is performed without sharing user data, then data privacy is protected, but network performance concerns arise
Solution Approach 1:
The patent creates and distributes copies of the neural network model to multiple local devices for independent training. Each device maintains a local copy of the model parameters and trains it locally on its own data. This copying approach enables privacy protection since no data is shared, while also improving network performance by distributing the computational workload and reducing network traffic to only essential model update exchanges.
Data Source
AI summary
A system having multiple devices that can host different versions of an artificial neural network (ANN). In the system, changes to local versions of the ANN can be combined with a master version of the ANN. In the system, a first device can include memory that can store the master version, a second device can include memory that can store a local version of the ANN, and there can be many devices that store local versions of the ANN. The second device (or any other device of the system hosting a local version) can include a processor that can train the local version, and a transceiver that can transmit changes to the local version generated from the training. The first device can include a transceiver that can receive the changes to a local version, and a processing device that can combine the received changes with the master version.