Adaptive Model Pruning for Federated Learning Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In federated learning, the massive number of vehicle models with varying degrees of divergence and low loss reduction can delay the convergence of a central model, and existing methods do not efficiently prune models to reduce computational and communication overhead.
Innovation Solution
An adaptive model pruning process that determines whether each vehicle's model loss reduction exceeds a predefined threshold, masking out models with low loss reduction and focusing on those with significant improvements, thereby selecting only the best candidates for training the global model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If all vehicle models are included in federated learning training, then the global model can leverage diverse data sources, but computational and communication overhead increases significantly
Solution Approach 1:
The patent extracts and removes vehicle models with low loss reduction from the federated learning training process. By calculating loss reduction for each vehicle model and comparing it against a threshold, the system identifies and excludes models that contribute minimally to global model improvement, thereby reducing computational and communication overhead while retaining models with significant learning contributions
Solution Approach 2:
The patent introduces loss reduction as a selection parameter to determine which vehicle models participate in training. By dynamically adjusting the threshold parameter, the system can control the number of participating models, balancing between diversity utilization and overhead reduction based on specific operational requirements
2Quantity of substance
If models with low loss reduction are included in training, then more data sources contribute to the global model, but convergence speed decreases
Solution Approach 1:
The patent removes models with low loss reduction from the training pool by comparing each model's loss reduction against a predefined threshold. This extraction process ensures that only models demonstrating significant learning improvements participate in training, thereby accelerating convergence without sacrificing the contribution of meaningful data sources
Solution Approach 2:
The patent applies partial action by selectively including only a subset of vehicle models in training based on their loss reduction performance. Rather than including all models, the system includes only those exceeding the threshold, achieving sufficient diversity while avoiding the convergence delays caused by including low-contributing models
3Productivity
If adaptive pruning based on loss reduction threshold is applied, then training efficiency improves, but some potentially valuable models may be excluded
Solution Approach 1:
The patent uses the loss reduction metric as a dynamic parameter to evaluate and select vehicle models for training. By adjusting the threshold parameter, the system can control the balance between efficiency and reliability, ensuring that models meeting the criterion are included while excluding those that do not, thereby maintaining training efficiency with reliable model selection
Data Source
AI summary
A system receives a plurality of data sets relating to differently trained versions of a global machine learning model, from a plurality of vehicles, the data sets including at least a present local loss value experienced by a current version of the global model executing on a given vehicle for which a data set of the plurality of data sets was received. The system determines a loss reduction for each received data set, representing a loss reduction since a previous local loss value included in a previous received data set corresponding to the given vehicle. The system determines whether the loss reduction for each received data set of the plurality of data sets exceeds a predefined threshold cutoff value and trains the global model using federated learning and based on the data sets of the plurality of data sets for which the loss reduction exceeds the predefined cutoff value.