A Method for Reducing Energy Consumption in Large-Scale Distributed Machine Learning Systems

Abstract

The invention discloses a method for reducing energy consumption of a large-scale distributed machine learning system. The invention classifies and identifies the load of the distributed machine learning system through a classifier, predicts the state, and reduces the energy consumption of the whole distributed machine learning system by reducing the communication between the distributed parameterservers and accelerating the task operation. The method of the invention comprises two parts: a machine learning load prediction and type identification method, and a distributed machine learning node parameter ''lazy synchronization'' mechanism. The invention can effectively reduce the waiting time of the system and accelerate the machine learning convergence rate by only transmitting significant updates to the remote data center to reduce the parameter synchronization mechanism of the wide area network communication. The prediction of machine learning load and the identification of machinelearning load types can help to improve the utilization rate of the working machine and avoid a large number of working machines in idle state after being turned on. The above method shortens the execution time of machine learning tasks, improves the utilization rate of the working machine, and greatly reduces the system energy consumption.

Description

technical field [0001] The invention relates to a method for reducing consumption of a large-scale computer system, in particular to a method for reducing energy consumption by rationally optimizing inter-computer communication and load scheduling in a large-scale distributed machine learning system. Background technique [0002] With the advancement of computing technology, communication technology, and sensor technology and the popularization of various smart terminal devices, more and more data are generated in human production and life, and the data growth rate is getting faster and faster. These rapidly generated raw data are generally large in scale, but the value density is low. The current common big data processing method is to introduce machine learning technology into the process of big data analysis and processing, and build a system model through linear regression, deep neural network and other methods And iterative training to mine the potential data rules and ...

AI Technical Summary

Problems solved by technology

However, in a heterogeneous WAN environment, especially when the links between geographically distant nodes are connected, the limited bandwidth will congest a large number of parameter updates, causing each iteration to take too long and the overall performance to decline significantly.

At the same time, a high delay will cause the working machines that depend on parameter updates to idle, resulting in waste of server power and increased energy consumption

Images