Distributed flow chart heterogeneous computing scheduling method

Abstract

The invention relates to the technical field of computers. The method is used for solving the problems of heterogeneous algorithm execution scheduling, data seamless circulation between heterogeneous algorithms, fine-grained division of parallel tasks, streaming data dynamic processing and the like in the current distributed cluster job processing flow. The invention provides a distributed flow chart heterogeneous computing scheduling method. The method comprises the steps of self-customrizing calculation flow chart configuration and then comprises the steps of manging computing process through centralized scheduling; carrying out hybrid scheduling on a heterogeneous algorithm; managing input and output data of a calculation program in the flow chart and forming a streaming data processing mechanism; providing a multi-language heterogeneous algorithm to access the SDK; configuring and executing a preprocessing algorithm program to process a data seamless circulation problem between heterogeneous algorithms, and configuring and executing the preprocessing algorithm program to carry out fine-grained division and the like on parallel tasks, thereby forming a complete solution for scheduling, executing and managing a heterogeneous algorithm computing process in a distributed cluster operation mode.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a method for dispatching and processing heterogeneous computing of distributed flow charts, in particular to a method for dispatching and managing large data heterogeneous algorithm processing processes. Background technique [0002] In recent years, with the diversification of data collection sources, the demand for big data processing in various industries has become increasingly obvious. Data production in the field of spatial data production basically requires multiple processes, and a variety of software algorithms are used for streamlined data processing. In view of the characteristics of large data volume, complex processing process and diversified processing tools in data production, it is necessary to adopt a process-based cluster computing model to improve the overall data production efficiency, and the practicability and feasibility of cluster computing are ma...

AI Technical Summary

Problems solved by technology

[0008] In order to solve the above problems, the present invention provides a distributed flowchart heterogeneous computing scheduling method and framework, the purpose of which is to provide a set of distributed flowchart heterogeneous computing scheduling framework and system to solve the process scheduling in the prior art The control granularity is not high, the streaming task data processing cannot be performed, the mixed scheduling of heterogeneous algorithms cannot be performed, and the data between heterogeneous algorithms cannot be seamlessly and automatically transferred

Images