Distributed computing method and system of cluster system and medium

Abstract

The invention discloses a distributed computing method and system of a cluster system and a medium. The method comprises steps of when the jobs are distributed, decomposing the jobs into a graph structure formed by a plurality of tasks; determining a to-be-executed target task, sending the to-be-executed target task to the cluster system for execution, waiting for execution result information of the target task, updating a graph structure if the execution is successful, and continuing to execute a new task until all tasks are executed if the graph structure still has no executed task; and if the execution fails, killing the task instance of the target task, performing a rollback operation on the target task according to the executed partial order dependency relationship until the target task is recovered to the initial state, and then re-executing the task. According to the method, the jobs can be decomposed in a transactional and reliable mode and distributed to the nodes in the cluster to be executed, the tasks of the nodes can be recovered and processed after the nodes are subjected to failures such as offline recovery and node restarting all the time, and therefore the requirements for high reliability, jobs transactional property, system expandability and maintainability of the cluster are met.

Description

technical field [0001] The invention relates to a cluster system and distributed computing technology, in particular to a distributed computing method, system and medium of a cluster system. Background technique [0002] Cluster technology is a computer system that uses multiple servers connected by cluster software to form a highly transparent large-scale server cluster. It provides services for clients as a whole, and clients can share all resources in the cluster network. At the same time, users of clients do not You need to care about which server the application is running on, and only care about whether the application can work continuously. From the perspective of the client, all the servers in the cluster are a system, like a large computer system, on which the application services required by the client are running. Distributed computing is a new computing method relative to centralized computing. The so-called distributed computing is to share information between...

AI Technical Summary

Problems solved by technology

At this stage, cluster operations may involve task allocation between different nodes, and the network of the cluster is unreliable, and problems such as node disconnection, network oscillation, and network delay are prone to occur

For simple task assignment in the cluster, it may be impossible to confirm whether the task has been executed, or the node cannot recover to the system state changes caused by job execution other than the log after the node goes offline or restarts and rejoins the cluster.

Images