Distributed task scheduling system and method

Abstract

The invention provides a distributed task scheduling system and method. The distributed task scheduling system comprises an interface service module, a message queue, a computing cluster and a core scheduling service module. The computing cluster is constructed by a plurality of service nodes through a point-to-point protocol, and the service nodes communicate with one another; the interface service module receives task request information sent by a client and sends the task request information to the core scheduling service module through the message queue; the core scheduling service module schedules the service nodes in the computing cluster by adopting a scheduling algorithm according to the task request information; the computing cluster selects a first target service node and a plurality of second target service nodes from the plurality of service nodes, receives connection request information sent by a new service node through the first target service node, verifies a secret key in the connection request information, and sends response information of connection success to the new service node if the verification is passed, therefore, the new service node establishes connection with the first target service node and the plurality of second target service nodes.

Description

technical field [0001] The invention relates to the field, in particular to a distributed task scheduling system and method. Background technique [0002] With the rapid development of the Internet, the functional complexity of applications is increasing day by day, and the computing power and storage capacity of a single server can no longer meet the demand. Therefore, a distributed cluster is formed by deploying multiple servers. The task scheduling system is an indispensable part of the distributed cluster, it can manage all the servers in the distributed cluster, and dispatch computing or storage tasks to a certain server according to the demand. [0003] In order to realize these functions, all servers in the cluster need to form a network through the task scheduling system, and can communicate with each other. The task scheduling system uses the http1.1 protocol or http2.0 protocol to form a network. When two servers communicate, at least one server needs to be confi...

AI Technical Summary

Problems solved by technology

Although it is possible to deploy proxy gateways to allow two servers without fixed public IPs to communicate, but when the number of servers in the cluster continues to increase, the pressure on a single proxy gateway will also increase. For stability reasons, it is necessary to Increase the number of proxy gateways, and the proxy gateway must be configured with a public IP, which increases the cost

Images