Unlock instant, AI-driven research and patent intelligence for your innovation.

Big data batch operation task scheduling method and device

A technology for batch operation and task scheduling, which is used in multi-programming devices, electrical digital data processing, program startup/switching, etc.

Pending Publication Date: 2020-12-04
BANK OF CHINA
View PDF3 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] In the embodiment of the present invention, a computer device is also provided to solve the technical problem that there is no large data batch job task scheduling mechanism developed based on Java in the prior art. And the computer program that can run on the processor, when the processor executes the computer program, realizes the above-mentioned big data batch job task scheduling method

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data batch operation task scheduling method and device
  • Big data batch operation task scheduling method and device
  • Big data batch operation task scheduling method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments and descriptions of the present invention are used to explain the present invention, but not to limit the present invention.

[0021] An embodiment of the present invention provides a large data batch job task scheduling method, figure 1 It is a flow chart of a large data batch job task scheduling method provided in the embodiment of the present invention, such as figure 1 As shown, the method includes the following steps:

[0022] S101, define a batch job for performing batch processing on big data, where the batch job includes: multiple tasks.

[0023] In specific implementation, in the above S101, one or more batch jobs can be defined according to the batch processing operations performe...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a big data batch operation task scheduling method and device, and the method comprises the steps: defining a batch operation for the batch processing of big data, wherein the batch operation comprises a plurality of tasks; configuring batch parameter information of the batch operation; executing each task in the batch operation according to the configured batch parameter information; monitoring an operation execution state of the batch operation and a task execution state of each task in the batch operation; and scheduling each task of the batch operation according to amonitoring result. According to the method, a big data batch operation task scheduling mechanism developed on the basis of a Java technology can be realized, so that batch processing tasks of the MPPdatabase can be scheduled.

Description

technical field [0001] The invention relates to the field of big data batch processing, in particular to a method and device for scheduling big data batch job tasks. Background technique [0002] This section is intended to provide a background or context to embodiments of the invention that are recited in the claims. The descriptions herein are not admitted to be prior art by inclusion in this section. [0003] The massively parallel analysis (Analytical Massively Parallel Processing, MPP) architecture can distribute tasks to multiple nodes in parallel, and after the calculation is completed on each node, the results of the respective parts are aggregated to obtain the final result. A database using the MPP architecture is called an MPP database. [0004] Because the MPP database has strong parallel data computing capabilities and massive data storage capabilities, it is very suitable for big data analysis and computing. growing mass of data. [0005] Big data analysis ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48G06F11/30
CPCG06F9/4806G06F11/3051
Inventor 杨晓晓
Owner BANK OF CHINA