Parameter Server Key-Value Consolidation for Distributed Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed machine learning systems face excessive storage overhead when handling multiple machine learning models simultaneously, due to the parameter server's linear storage growth and high computational and network bandwidth requirements.
Innovation Solution
A distributed system where a parameter server maintains parameters as key-value pairs, saving multiple key-value pairs with the same key as a single key corresponding to multiple values, allowing computing devices to execute training or prediction algorithms in parallel, while reducing storage, computational, and network overhead by merging and compressing parameter operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the parameter server stores parameters for multiple machine learning models simultaneously, then the system can execute training and prediction for multiple models, but the storage overhead grows linearly
Solution Approach 1:
The patent merges parameters from multiple machine learning models that share the same key into a single consolidated storage structure. Instead of storing parameters for each model separately, the system combines them by key, allowing the parameter server to serve multiple models while reducing redundant storage. This directly addresses the linear storage growth problem by eliminating duplication across models.
Solution Approach 2:
The parameter server is designed to handle multiple machine learning models universally through a key-based parameter storage mechanism. By organizing parameters with shared keys, the system enables a single parameter server instance to support multiple models simultaneously, improving versatility without proportionally increasing storage overhead.
2Productivity
If the parameter server maintains parameters for multiple machine learning models, then concurrent training and prediction operations can be performed, but the computational overhead increases
Solution Approach 1:
The system merges parameter access operations for multiple models by consolidating parameters with the same key. This reduces the computational overhead of parameter management while maintaining the ability to perform concurrent training and prediction operations across multiple models, as the parameter server can efficiently serve multiple requests from a unified parameter structure.
Solution Approach 2:
The parameter storage is segmented by key rather than by model, allowing the system to efficiently manage parameters for multiple models. This segmentation strategy enables the parameter server to handle concurrent operations by organizing parameters in a way that minimizes computational overhead during read and write operations.
3Productivity
If the parameter server stores parameters for multiple machine learning models, then the system supports high concurrency, but the network bandwidth occupation increases
Solution Approach 1:
The system merges parameter transmissions by consolidating parameters with identical keys from multiple models into single network transmissions. This reduces network bandwidth occupation while maintaining high concurrency support, as the parameter server can efficiently transmit shared parameters to multiple computing devices without redundant network traffic.
Data Source
Figure 1~3
Figure 4~5
Figure 6~7
AI summary
Provided are a distributed system for executing machine learning and a method therefor. The distributed system comprises: a parameter server, for maintaining parameters of a plurality of machine learning models, the parameters being in the form of a key-value pair, the parameter server saving a plurality of key-value pairs having the same key in the form of a single key corresponding to a plurality of values; and a plurality of computing devices, configured to execute, in parallel, algorithms concerning the plurality of machine learning models.