Cloud workflow intelligent management and scheduling system in cloud computing platform

Abstract

The invention discloses a cloud workflow intelligent management and scheduling system in a cloud computing platform, which comprises a system front end, a scheduling controller, a cloud workflow scheduler and a resource distributor. The system front end uploads a cloud workflow file; the scheduling controller generates information for starting the cloud workflow scheduler after receiving the cloud workflow file; the resource distributor receives the information of starting the cloud workflow scheduler and then obtains residual resources through the container management platform; the container management platform creates a plurality of cloud workflow schedulers on the node after receiving the request information for creating the cloud workflow schedulers; and each cloud workflow scheduler determines a total resource configuration demand according to the cloud workflow file, receives residual resources, generates a cloud workflow computing node according to the total resource configuration demand and the residual resources, and processes the cloud workflow file. The scheme disclosed by the invention can be arranged on a large-scale distributed cluster, the cloud workflow receiving proportion can be improved by 20%, and workflow task management and scheduling can be efficiently and reliably completed in a multi-load environment.

Description

technical field [0001] The invention relates to the field of resource scheduling and management, in particular to a cloud workflow intelligent management and scheduling system in a cloud computing platform. Background technique [0002] Cloud computing and big data are major changes and inevitable trends in the development of informatization. They are the commanding heights of international competition in the information age and the fuel for new kinetic energy in economic development. Nowadays, cloud resources and cloud workflow management and scheduling in the cloud control system platform have unique challenges, such as complex and uncertain workflow scheduling process, low resource utilization, high data exchange, and high timeliness requirements. In order to meet the quality of service requirements of users on the existing cloud computing platform, cloud service providers have to adopt the method of overproviding resources, which causes a great waste of resources. In ad...

AI Technical Summary

Problems solved by technology

Nowadays, there are unique challenges in cloud resource and cloud workflow management and scheduling in the cloud control system platform, such as complex and uncertain workflow scheduling process, low resource utilization, high data exchange, and high timeliness requirements.

In order to meet the service quality requirements of users on the existing cloud computing platform, cloud service providers have to adopt the method of overproviding resources, which causes a great waste of resources

In addition, existing cloud computing platforms have natural barriers to running on ultra-large-scale clusters. With the increase of cluster size and load, its scheduling and execution completion time increase rapidly, and the number of corresponding schedulable tasks will decrease rapidly, thus reducing cloud workload. Flow file receiving ratio

Images