A method and system for job scheduling in distributed data processing system with identification of optimal network topology

Abstract

The method of the present invention provides an automatic and optimised selection of the network topology for distributing scheduling of jobs on the computers of the modified network topology. The automatic and optimised selection of the network topology starts from the current topology and a desired number of additional connections. In this way the method of the present invention provides a higher convergence speed for the modified consensus algorithm in comparison e.g. to a simple ring network. The method exploits the so called small-world networks. Small-world networks are more robust to perturbations than other network architectures. The preferred embodiment provides a workload scheduling system which is highly scalable to accommodate increasing workloads within a heterogeneous distributed computing environment. A modified average consensus algorithm is used to distribute network traffic and jobs amongst a plurality of computers.

Description

technical field [0001] The present invention relates to the field of computer networks, and more specifically, the present invention relates to a job scheduling system in a distributed computing environment. Background technique [0002] Job scheduling and workload balancing across multiple network-connected resources are increasingly important tasks in IT environments. Many grid computing environments are driven by scheduling work across distributed sets of resources (eg, computing, storage, communication capabilities, software licenses, special equipment, etc.). In essence, scheduling is an optimization problem, which is very simple when only one resource type is involved. However, while further performance improvements can be achieved by including more resource variables in the scheduling process, the resulting multivariate optimization becomes a difficult mathematical problem. [0003] Existing job scheduling systems generally adopt a master / agent (master / agent) archit...

AI Technical Summary

Problems solved by technology

The disadvantage of the above method is that the user should build a virtual network including the logical topology of the computer

In other words, it is the user's responsibility to choose the correct topology, and this can lead to wrong choices that jeopardize network efficiency

Images