Big data task scheduling management method, device and apparatus and storage medium

A technology of task scheduling and management methods, which is applied in the field of big data task scheduling management methods, devices, equipment, and computer-readable storage media, can solve the problems of inability to uniformly schedule and manage batch tasks, and low timeliness of large-batch data processing, so as to improve The effect of processing timeliness, improving the efficiency of operation and maintenance management, and reducing the difficulty of operation

Pending Publication Date: 2020-05-08
CHINA PING AN PROPERTY INSURANCE CO LTD
View PDF0 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The main purpose of the present invention is to provide a big data task scheduling management method, device, equipment and computer-readable storage medium, aiming to solve the problem that the existing big data task scheduling management technology cannot perform unified scheduling management on batch tasks, resulting in large batch Technical issues with low data processing timeliness

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data task scheduling management method, device and apparatus and storage medium
  • Big data task scheduling management method, device and apparatus and storage medium
  • Big data task scheduling management method, device and apparatus and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0075] It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0076] The invention provides a big data task scheduling management device.

[0077] refer to figure 1 , figure 1 It is a schematic structural diagram of the hardware operating environment of the device involved in the embodiment solution of the big data task scheduling management device of the present invention.

[0078] Such as figure 1 As shown, the big data task scheduling management device may include: a processor 1001 , such as a CPU, a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 . Wherein, the communication bus 1002 is used to realize connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standar...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of big data, and discloses a big data task scheduling management method, which comprises the steps of reading a task scheduling configuration file; extracting running information of all tasks from the task scheduling configuration file; based on the operation information, determining a dependency relationship among the tasks; based on the dependency relationship, integrating the tasks into a task chain with a front-back association relationship; drawing a tree-shaped flow chart corresponding to the tasks based on the task chain and displaying the tree-shaped flow chart; and performing visual scheduling management on each task based on the tree-shaped flow chart. The invention further discloses a big data task scheduling management device and apparatus and a computer readable storage medium. According to the invention, a plurality of big data tasks are scheduled and managed in a unified manner, and the processing timeliness of large-batch data is improved.

Description

technical field [0001] The present invention relates to the field of big data technology, in particular to a big data task scheduling management method, device, equipment and computer-readable storage medium. Background technique [0002] In the prior art, big data systems usually use professional tools (such as oozie) for multi-task scheduling, and the test and production environments can only be viewed and operated remotely through the bastion host, and only a single task can be managed at a time, such as viewing The specific configuration of the task, the scheduling period, the running time of the script under each application, and the running status of the pre-dependency must be manually searched by the operation and maintenance personnel in the corresponding task pop-up window, which is time-consuming and laborious. [0003] Usually, there are tens of thousands of tasks running at the same time in a big data system, and there is usually a certain relationship between di...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/901G06F9/48
CPCG06F9/4843G06F16/9024
Inventor 卢显锋
Owner CHINA PING AN PROPERTY INSURANCE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap