Federated Neural Network Training with Local Sample Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Multiple data owners in industries like e-commerce and finance face challenges in jointly training neural network models due to privacy concerns, as they are unwilling to share their private data, leading to poor model performance and diversity when using non-independent and identically distributed data sets.
Innovation Solution
A method where each data owner trains a neural network submodel using their private data, shares model update information and local sample distribution with a second member device, which reconstructs a neural network model and allocates it back, allowing for personalized updates based on local and overall sample probability distributions, ensuring data privacy and improving model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If data owners share their private data for joint training, then model performance is improved, but data security and privacy are compromised
Solution Approach 1:
The patent extracts only the necessary model update information (gradients, parameters) from the training process and transmits them to the server, while keeping the raw private data localized and never leaving the data owner's device. This extraction approach allows model improvement without compromising data privacy.
Solution Approach 2:
The patent introduces a server as an intermediary that coordinates the federated learning process. The server aggregates model updates from multiple data owners without accessing their private data, enabling collaborative model training while maintaining data security through this intermediate coordination layer.
2Object-affected harmful factors
If data owners use their local private data for training, then data security is maintained, but model diversity and overall performance deteriorate due to non-IID data distribution
Solution Approach 1:
The patent merges model updates from multiple data owners through server aggregation to create a diverse and robust global model. By combining gradient information and parameters from different sources with varying data distributions, the system achieves model diversity without requiring access to individual private datasets.
Solution Approach 2:
The patent modifies model parameters through federated aggregation rather than changing the data itself. The server adjusts model parameters by combining updates from multiple data owners, effectively adapting the model to handle non-IID data distributions while maintaining data security at each participant's location.
3Manufacturing precision
If centralized training is used to improve model performance, then model accuracy is enhanced, but training efficiency and coordination complexity increase
Solution Approach 1:
The patent segments the centralized training process into distributed local training tasks performed by individual data owners. Each participant trains the model locally on their own data, eliminating the need to centralize large datasets and improving training efficiency through parallel processing while maintaining model accuracy.
Solution Approach 2:
The patent implements a dynamic federated learning process where model training occurs in iterative rounds with flexible participation. Data owners can join or leave the training process dynamically, and the server adapts aggregation strategies based on available participants, enhancing both efficiency and adaptability compared to rigid centralized approaches.
Data Source
AI summary
A method for training a neural network model includes: at each first member device, obtaining predicted label data according to a first neural network submodel of the first neural network submodel by using private data for model training, and determining model update information of the first neural network submodel according to the predicted label data and real label data; providing, by each first member device, the model update information of the first neural network submodel and local sample distribution information to a second member device; at the second member device, performing neural network model reconstruction, determining an overall sample probability distribution, and allocating a reconstructed neural network model and the overall sample probability distribution to each first member device; and updating the first neural network submodel at each first member device according to a local sample probability distribution, the reconstructed neural network model, and the overall sample probability distribution.