Distributed Stochastic Gradient Descent for Telemetry Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for collecting and analyzing telemetry data to determine resource health or user engagement issues are inefficient due to sequential processing methods, leading to prolonged computation times and resource-intensive data transmission.
Innovation Solution
Implementing a parallel stochastic gradient descent (SGD) system that computes local models in distributed geographic locations, transmitting these models instead of large training datasets, and combining them into a global model using symbolic representations to determine resource or user performance issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sequential stochastic gradient descent is used to process training data, then model parameter computation is accurate, but computation time is excessively long
Solution Approach 1:
The training dataset is divided into multiple partitions distributed across different geographic locations. Each location independently computes local model parameters using parallel SGD on its local data partition, then these local models are combined through symbolic representations to produce the final global model parameters. This segmentation enables concurrent processing while maintaining computational accuracy.
2Reliability
If large training datasets are transmitted across the network, then model computation can be performed centrally, but network bandwidth consumption and resource usage increase significantly
Solution Approach 1:
Instead of transmitting the entire large training dataset across the network, the system extracts and transmits only the computed local model parameters and symbolic representations from each geographic location. This extraction approach maintains model computation reliability while dramatically reducing network bandwidth consumption and resource usage.
3Productivity
If parallel processing is implemented across distributed locations, then computation time is reduced, but system complexity increases
Solution Approach 1:
Symbolic representations serve as intermediaries that bridge the parallel computations performed at different geographic locations. These symbolic representations encode the relationships between local model parameters and enable their combination into global model parameters without requiring complex coordination or communication protocols, thus managing system complexity while maintaining high computation speed.
Data Source
AI summary
Described herein is a system that transmits and combines local models, that individually comprise a set of local parameters computed via stochastic gradient descent (SGD), into a global model that comprises a set of global model parameters. The local models are computed in parallel at different geographic locations along with symbolic representations. Network transmission of the local models and the symbolic representations, rather than transmission of the large training data subsets processed to compute the local models and symbolic representations, conserves resources and decreases latency. The global model can then be used as a model to determine a likelihood of a monitored resource or a user of the monitored resource experiencing a problem with respect to performance or completion of one or more operations. The system can also implement an action to assist in resolving or avoiding the problem.