A method and system for automatic task parallelism suitable for distributed machine learning

Abstract

The present invention provides an automated task parallel method and system suitable for distributed machine learning, which solves the defects of the existing distributed machine learning programming interface: only providing the read-write interface of key-value pairs leads to system data access behavior and application logic Tightly coupled. This defect intensifies competition for network bandwidth resources in distributed clusters, making it difficult for programmers to parallelize tasks. The system of the present invention includes a working node module, a service node module, a main node module, a tensor module, a scheduling module, a message tracking module, a stage module, a stage group module and an execution engine module. The present invention decouples the read-write access behavior and the logic of the application program by providing a higher-level programming abstraction. The runtime system firstly performs dynamic task division according to the load of the service node, and secondly executes the machine learning tasks automatically and in parallel, greatly Reduce the burden on programmers to write high-concurrency machine learning applications.

Description

technical field [0001] The invention belongs to the cross-technical field of distributed computing and machine learning, and in particular relates to a parallel method and system for automatic tasks suitable for distributed machine learning. Background technique [0002] As a traditional method of mining data value, machine learning algorithms are widely used in fields such as natural language processing, text analysis, speech recognition, automatic driving of motor vehicles, and bioinformatics. With the advent of the era of big data, the value of data is becoming more and more prominent, especially the commercial value contained in it, so machine learning has been valued. However, as the scale of data and the corresponding model parameters that need to be learned become larger and larger, a single computing node cannot meet the needs of large-scale machine learning due to its limited memory resources, computing resources, and memory access bandwidth resources. Distributing...

AI Technical Summary

Problems solved by technology

The above method and system can effectively solve the network transmission bottleneck problem in the existing distributed machine learning system and improve the system concurrency, thereby improving the overall performance of the system

Images