Distributed On-Device Learning With Temporal Region Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Federated learning systems face challenges in training machine learning models due to non-representative data samples, as device availability is often correlated with geocultural boundaries and time zones, leading to uneven distribution of training data across different regions.
Innovation Solution
The approach involves subdividing the world into regions based on temporal availability patterns, allowing for consistent data sampling within each region and using multitask learning to combine data from multiple regions, ensuring that each region's model benefits from diverse training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If federated learning is performed using all available devices globally, then the model can be trained on large amounts of data, but the data sample becomes non-representative due to temporal availability correlations with time zones and geocultural boundaries
Solution Approach 1:
The patent divides the global device population into multiple regions based on temporal availability patterns and time zones. Each region is trained separately with its own model copy, ensuring that the data sample within each region is more representative. This segmentation resolves the contradiction by maintaining large overall data quantity while improving representativeness through regional subdivision.
Solution Approach 2:
The patent applies local quality by training region-specific models that are optimized for local data characteristics. Each region's model is trained on locally available data during local time zones, ensuring the model quality is tailored to local conditions. This improves representativeness while maintaining large overall data quantity across all regions.
2Productivity
If devices are selected for training during specific time zones, then data availability increases, but the model becomes biased toward specific geocultural boundaries and languages
Solution Approach 1:
The patent segments the global model into multiple region-specific model copies, each trained during its local time zone when devices are most available. This segmentation allows each region to benefit from high productivity during its active hours while maintaining adaptability through the presence of other regional models that can be combined or used for comparison.
Solution Approach 2:
The patent merges results from multiple region-specific models to create a comprehensive solution. By combining insights from different regional models trained on diverse data, the system achieves both high productivity (each region trains when active) and versatility (combined model serves global population).
Data Source
AI summary
The present disclosure provides systems and methods for distributed training of machine learning models. In one example, a computer-implemented method is provided for training machine-learned models. The method includes obtaining, by one or more computing devices, a plurality of regions based at least in part on temporal availability of user devices; selecting a plurality of available user devices within a region; and providing a current version of a machine-learned model associated with the region to the plurality of selected user devices within the region. The method includes obtaining, from the plurality of selected user devices, updated machine-learned model data generated by the plurality of selected user devices through training of the current version of the machine-learned model associated with the region using data local to each of the plurality of selected user devices and generating an updated machine-learned model associated with the region based on the updated machine-learned model data.