A distributed task scheduling method and system

A technology of distributed tasks and scheduling methods, applied in the field of distributed task scheduling methods and systems, which can solve the problems of difficult maintenance of task dependencies and difficulty in expanding new functions, and achieve the effect of convenient management and high availability

Active Publication Date: 2018-12-07
ZHONGAN ONLINE P&C INSURANCE CO LTD
View PDF8 Cites 19 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problems of the prior art, the embodiment of the present invention provides a distributed task scheduling method and syst

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A distributed task scheduling method and system
  • A distributed task scheduling method and system
  • A distributed task scheduling method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0055] Such as figure 1 as shown, figure 1 It is a flowchart of the distributed task scheduling method disclosed in Embodiment 1 of the present invention, and the method includes the following steps:

[0056] S1: After the client server is started, it automatically scans all tasks, establishes a link with the intermediate server, and reports its own information and task information to the intermediate server.

[0057] Specifically, after the client server is started, it automatically scans all tasks, establishes a TCP connection with the intermediate server, and reports its own information and task information to the intermediate server, wherein the client server's own information includes its IP address, etc., and the task information includes task configuration information , task ID, task execution time, etc. The task ID is used to uniquely identify a task, which can be the number of the task or other flag items. It should be noted that, in this embodiment, there are multi...

Embodiment 2

[0084] Such as Figure 5 as shown, Figure 5 It is a schematic structural diagram of the distributed task scheduling system disclosed in Embodiment 1 of the present invention, and the system includes:

[0085] A client server, wherein there are multiple client servers, which are used to scan tasks, report task information and configuration to an intermediate server, and execute assigned subtasks;

[0086] Intermediate server (broker), there are multiple intermediate servers, and the intermediate servers are connected to each other, and are used to manage the client server, arrange the process of multiple tasks, and split a task into multiple independent After the subtasks are assigned to each client server, the dependent call of the forwarding task;

[0087] A distributed coordination system (Zookeeper), connected to the intermediate server, for uniformly coordinating the intermediate server;

[0088] Further, the system also includes:

[0089] A back management system (Da...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed task scheduling method and system, belonging to the computer technical field. The method comprises the following steps: S1, after the client server starts up, alltasks are automatically scanned, a link is established with the intermediate server, and self-information and task information are reported to the intermediate server; S2, that intermediate server obtains the client server information and the task information, and splits a task into a plurality of independent sub-tasks according to the task configuration information, and distribute the task to each client server; S3, the client server executes the sub-tasks assigned by the intermediate server. Through integration of the task scheduling engine in the intermediate server, the dependency relationship of the tasks are conveniently managed. Through the intermediate server, a task is divided into several independent sub-tasks, and the client server is called to execute the assigned task items in parallel to achieve parallel scheduling.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a distributed task scheduling method and system. Background technique [0002] Distributed task scheduling refers to dividing an application or business into multiple tasks and handing them over to multiple servers for processing. Distributed task scheduling can solve the problem of insufficient resources of a single server and achieve the purpose of large-scale data processing. [0003] There are many distributed frameworks. Among them, the traditional Zookeeper distributed application coordination service fragments the data and assigns it to multiple application servers to perform processing to achieve parallel execution, thereby improving the processing speed. However, since Zookeeper needs to manage many application servers, the start and end records of each task also need to be written into Zookeeper, resulting in a relatively heavy burden on Zookeeper, and if you want to...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/48
CPCG06F9/4881
Inventor 熊杰付本成
Owner ZHONGAN ONLINE P&C INSURANCE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap