Completion time optimization-oriented scientific workflow scheduling method and system

A technology for completing time and scheduling methods, applied in the computer field, can solve problems such as lack of solutions in the system, prolonging the length of workflow scheduling, resource adjustment, etc.

Pending Publication Date: 2020-10-30
SHANGHAI JIAO TONG UNIV
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

A large number of known workflow systems will put workflow tasks into some preset virtual machines, containers, or their custom workflow runtimes to run, but they cannot be executed during the running of workflow tasks. Resource adjustment, so that in the case of multiple workflow tasks running at the same time, it may lead to an imbalance of resources allocated between workflow tasks, extending the scheduling length of one or more workflows
For scientific workflows with heterogeneous tasks, how to dynamically optimize the resource allocation of long or short tasks, the existing system lacks an efficient solution

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Completion time optimization-oriented scientific workflow scheduling method and system
  • Completion time optimization-oriented scientific workflow scheduling method and system
  • Completion time optimization-oriented scientific workflow scheduling method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0079] The present invention will be described in detail below in conjunction with specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0080] This application proposes a technical solution for running scientific workflows based on serverless architecture, in which container technology is used to schedule workflows under the framework of Kubernetes and Knative, so as to achieve higher overall completion time and cluster performance. Therefore, this technical solution can be used as a scientific workflow execution platform to execute various scientific workflows including heterogeneous tasks.

[0081] In this solution, in order to solve ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a completion time optimization-oriented scientific workflow scheduling method and system. The method comprises the steps: converting a scientific workflow task into a server-free function and deploying the server-free function into a corresponding cluster; converting a given scientific workflow into a corresponding directed acyclic graph; for each layer of tasks in the directed acyclic graph, allocating resources to the tasks and running the tasks according to parameter configuration; in the task running process, keeping the monitoring of the cluster, and dynamically adjusting resource allocation of each task. Compared with the prior art, the method has the advantage that higher overall completion time and cluster performance are obtained by fully utilizing the elastic telescopic capacity provided by the server-free framework.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a scientific workflow scheduling method and system based on serverless architecture and oriented to completion time optimization. Background technique [0002] In scientific computing, scientific workflow is a widely used abstraction. [0003] A workflow is generally represented by a directed acyclic graph (DAG), including a series of tasks and data input or output within or between tasks. Building a scientific application as a workflow provides a convenient abstraction to represent scientific problems, sinking complex parallel scheduling into the workflow scheduling system. [0004] Reproducibility of work is critical for scientific workflows. A scientific workflow task may have multiple dependencies or multiple configuration files. Because of these complex dependencies and configuration files, there may be dependency conflicts between different workflow tasks. How ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06Q10/10G06F9/50
CPCG06F9/5016G06F9/5038G06Q10/103
Inventor 钱诗友周杰薛广涛
Owner SHANGHAI JIAO TONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap