Unlock instant, AI-driven research and patent intelligence for your innovation.

Client management method, device and system based on workflow task scheduling system

A technology of task scheduling and management method, which is applied in the field of client management based on workflow task scheduling system, can solve problems such as not supporting client verification, and achieve the effect of wide application prospects, highlighting substantive features, and reliable design principles

Pending Publication Date: 2022-05-06
SUZHOU LANGCHAO INTELLIGENT TECH CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Aiming at the problem that the existing DolphinScheduler framework does not support the existence verification of the client when running tasks, the present invention provides a client management method, device, and system based on a workflow task scheduling system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Client management method, device and system based on workflow task scheduling system
  • Client management method, device and system based on workflow task scheduling system
  • Client management method, device and system based on workflow task scheduling system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0062] DolphinScheduler is a distributed and easily scalable visual DAG workflow task scheduling system. As a top-level project of Apache, DolphinScheduler is the same as the workflow task scheduling system in this application;

[0063] The DolphinScheduler architecture includes the following:

[0064] UI and API: interactive modules, configuration management DAG, file resources required for management task operation, and data source monitoring.

[0065] Master cluster: analyze and split the DAG, and distribute the tasks in the DAG to the Workers. The Master is the same as the main management terminal in this application.

[0066] Worker cluster: Download the resource files needed to execute tasks from the file system, execute specific tasks, monitor the status of tasks and report to the Master. Worker is the same as the task execution end in this application.

[0067] Tasks: Support running various tasks such as Flink, MR, Spark, etc. in the cluster. Through the combinati...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of distributed system management, and particularly provides a client management method, device and system based on a workflow task scheduling system.The method comprises the following steps that whether a client is installed on a task execution end or not is judged according to environment variables of the system, and a judgment result is reported and displayed; when it is judged that the client is not installed on the task execution end, client installation is carried out in the mode of downloading packet decompression configuration environment variables or in the mode of a yum source; checking whether a client on the task execution end meets task operation or not; and when it is found through verification that the client does not meet the condition, the task execution end group is replaced, or the client is installed for the task execution end which does not meet the condition. In the aspect of workflow execution, before execution, verification is carried out according to the Worker condition in the Worker group, and the task failure caused by the fact that the client is not installed is avoided.

Description

technical field [0001] The invention relates to the technical field of distributed system management, in particular to a client management method, device and system based on a workflow task scheduling system. Background technique [0002] DolphinScheduler is a distributed and easily scalable visual DAG workflow task scheduling system. A top-level project for Apache. The DolphinScheduler component can install multiple Workers. When executing tasks, it will distribute tasks to different Workers for execution by polling or load balancing according to the running status of the Worker host. It also supports Worker group management. This method divides Workers into different groups. When executing a task, by selecting the corresponding Worker group, the task can be run in the corresponding Worker group. [0003] In the existing DolphinScheduler framework, when running a task, it does not support the existence check of the client. When running tasks such as Flink, Spark, MR, and...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/50
CPCG06F9/5083Y02D10/00
Inventor 胡永泽
Owner SUZHOU LANGCHAO INTELLIGENT TECH CO LTD