Federated Learning Edge Device Training Data Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning on edge devices is computationally expensive and time-consuming due to the need to process all samples, which is particularly challenging for IoT devices with limited computation power and memory.
Innovation Solution
Edge devices selectively choose training dataset items based on the current federated model to filter out redundant data samples, reducing the number of samples trained on and thereby decreasing computational burden without requiring additional synchronization or sensitive information exchange.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If all training samples are processed during each iteration, then training completeness is improved, but computational cost and time consumption increase significantly
Solution Approach 1:
The patent applies partial action by selectively processing only a subset of training samples that are most beneficial for model improvement. Instead of processing all samples uniformly, the system identifies and processes only those samples that will contribute most to reducing the loss function, thereby achieving effective training with reduced computational overhead
Solution Approach 2:
The patent dynamically changes the parameter set being trained on by selecting different subsets of training samples based on the current model state. The selection criteria and subset composition are adjusted iteratively based on performance metrics, allowing the system to adaptively optimize which parameters (samples) are processed at each iteration
2Productivity
If the size of training dataset is reduced by filtering, then computational burden is decreased, but training effectiveness may be compromised
Solution Approach 1:
The patent changes the parameters being optimized by dynamically selecting which training samples to process based on their expected contribution to model improvement. The selection is adapted iteratively as the model evolves, ensuring that the most valuable samples are processed while less valuable ones are filtered out, maintaining training effectiveness with reduced computational load
Solution Approach 2:
The system uses feedback from model performance metrics (such as loss values) to guide the selection of training samples. By monitoring how the model performs on different samples and adjusting the selection criteria accordingly, the system ensures that filtering does not compromise training effectiveness while still achieving computational efficiency
3Reliability
If federated learning is implemented across multiple edge devices, then data privacy is improved, but communication overhead and coordination complexity increase
Solution Approach 1:
The patent extracts the coordination and aggregation functions from the distributed edge devices and centralizes them in a cloud server. This extraction simplifies the complexity at individual edge devices while maintaining the privacy benefits of federated learning, as the server handles the complex coordination without accessing sensitive local data
Data Source
AI summary
Some embodiments are directed to a federated learning system. A federated model is trained on respective local training datasets of respective multiple edge devices. In an iteration, an edge device obtains a current federated model, determines a model update for the current federated model based on the local training dataset, and sends out the model update. The edge device determines the model update by applying the current federated model to a training input to obtain at least a model output for the training input; if the model output does not match a training output corresponding to the training input, include the training input in a subset of filtered training inputs to be used in the iteration; and determining the model update by training the current federated model on only the subset of filtered training inputs.