Double-acceleration distributed asynchronous optimization method based on Nesterov gradient method and heavy ball method

Abstract

The invention discloses a double-acceleration distributed asynchronous optimization method based on a Nesterov gradient method and a heavy ball method. The double-acceleration distributed asynchronousoptimization method comprises the following steps of initializing variables, determining a time delay value and an activation node, clearing out outdated information, building a network, determiningparameters and updating the variables. In the invention, a very universal asynchronous model is adopted. In the model, 1) all agents can communicate with their neighbors at any time and without any coordination control; and 2) by using the time delay information, all the agents can locally process their own sub-problems. Therefore, the asynchronous algorithm can greatly reduce idle time of a communication link, relieve network and memory access congestion and save power, and has higher robustness and fault tolerance for an actual network system. The method is widely applied to large-scale machine learning and network information processing.

Description

technical field [0001] The invention relates to the technical field of distributed optimization of communication data, in particular to a dual-acceleration distributed asynchronous optimization method based on the Nesterov gradient method and the weighted ball method. Background technique [0002] In the early days, industrial data was mostly small in size and low in dimensionality, and the data was often stored in an intelligent body system, which could effectively process these simple data only by relying on centralized algorithms. However, as the types of industrial data increase, the data is often stored in multiple agent systems, and the data becomes complex and difficult to process. Centralized algorithms have been difficult to apply to process industrial data. Researchers began to work on the study of distributed algorithms. There are two classic architectures in current systems: master-slave computing architecture and distributed computing architecture. We take fou...

AI Technical Summary

Problems solved by technology

[0008] (1) In the actual environment of large-scale problems and networks, the distributed synchronization algorithm has defects such as insufficient data processing capacity, poor robustness, low utilization of network links, and low fault tolerance due to the requirement of a global clock.

[0009] (2) The existing asynchronous algorithm cannot effectively solve the delay problem, and data packet loss cannot be avoided in an asynchronous communication network

At the same time, most asynchronous algorithms can only converge to the optimal solution at a sublinear speed

[0010] (3) Although synchronous algorithms analyze individual acceleration technologies, existing asynchronous algorithms ignore the discussion of acceleration technologies

In addition, neither the synchronous algorithm nor the asynchronous algorithm discusses whether the two acceleration technologies can be effectively combined to achieve double acceleration.

[0011] It can be seen that the conventionally used algorithms in the prior art can no longer meet people's requirements

Images