SGD (Stochastic Gradient Descent) algorithm optimization system and method

Abstract

The invention disclsoes an SGD (Stochastic Gradient Descent) algorithm optimization system and method. The system comprises: a plurality of distributed calculation nodes based on an FPGA (Field Programmable Gate Array) and a center calculation node based on a CPU (Central Processing Unit), wherein each distributed calculation node is used for carrying out parallel calculation on data to be processed in an SGD algorithm through a corresponding parallel data path in the FPGA; the center calculation node is used for distributing and dispatching a data processing task. The center calculation node can be used for distributing different calculation tasks according to structural characteristics of each distributed calculation node, and a calculation performance, an energy efficiency ratio and a computation real-time performance of each distributed calculation node are improved; all the distributed calculation nodes do not need to be subjected to data exchange and network overhead of a calculation cluster is reduced. The distributed calculation nodes are arranged based on the FPGA and the center calculation node is arranged based on the CPU, so that a heterogeneous calculation platform is formed; the SGD algorithm is subjected to parallel designing and the SGD algorithm under a Spark framework is extremely optimized; the whole performance of processing massive data by the SGD algorithm is improved.

Description

technical field [0001] The invention relates to the field of big data processing, in particular to an SGD algorithm optimization system and method. Background technique [0002] With the development of information technology, we have entered the era of big data. Many machine learning algorithms can be transformed into convex function optimization problems, that is, the task of finding the minimum value of a regression function, the simplest method of which is gradient descent. The stochastic gradient descent (SGD) algorithm is a typical algorithm for convex function optimization problems in machine learning algorithms. [0003] In the Spark big data processing framework, the processing idea of ​​the SGD algorithm is: first randomly initialize the gradient value and weight value of the function, and use parameters to specify a random sampling subset of the full set of data, and then calculate the average value of the gradient of the data points in the subset , that is, get ...

AI Technical Summary

Problems solved by technology

However, under the Spark big data platform, the data set sampling and gradient calculation of the SGD algorithm adopt parallel operations, so that each computing node server needs to undertake a large amount of data calculation, and in a distributed computing environment, each computing node needs to perform Data exchange (shuffle operation) to update the random gradient value and weight value increases the network overhead of the computing cluster

Images