Job scheduling method, device and distributed system

A distributed system and job scheduling technology, applied in the field of data processing, can solve the problems of insufficient utilization of cluster resources, large amount of data in central nodes, affecting scheduling efficiency, etc., to improve resource utilization, reduce job running time, improve The effect of scheduling efficiency

Active Publication Date: 2019-12-06
ALIBABA GRP HLDG LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, in this job scheduling method, the amount of data processed and maintained by the central node is huge, which affects the scheduling efficiency, and does not make full use of cluster resources, resulting in waste of resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Job scheduling method, device and distributed system
  • Job scheduling method, device and distributed system
  • Job scheduling method, device and distributed system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0057] The implementation of the present application will be described in detail below with reference to the accompanying drawings and examples, so as to fully understand and implement the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects.

[0058] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0059] Memory may include non-permanent storage in computer readable media, in the form of random access memory (RAM) and / or nonvolatile memory such as read only memory (ROM) or flash RAM. Memory is an example of computer readable media.

[0060] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of a progr...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

A job scheduling method including notifying, by a first control node scheduling a first task, a second control node scheduling a second task to obtain execution data generated by at least one task instance of the first task when the at least one task instance of the first task completes execution; obtaining, by the second control node, the execution data generated by the at least one task instance of the first task and assigning the execution data to each task instance of the second task; and scheduling, by the second control node, an execution of at least one task instance of the second task and processing the execution data. The techniques of the present disclosure improve scheduling efficiency and resource utilization rate.

Description

technical field [0001] The present application belongs to the technical field of data processing, and in particular relates to a job scheduling method, device and distributed system. Background technique [0002] In traditional distributed systems, distributed jobs include a limited number of tasks (tasks), and there are certain dependencies between tasks. Each task is divided into multiple task slices (instances). When the job is running, each Task slices of tasks perform data processing. [0003] Based on the traditional distributed system, the job scheduling method is usually performed by the central node (master). For example, suppose the job has two tasks, namely task1 and task2. The dependency between task1 and task2 is task2 and the input is task1. output, when the job runs, the central node will look for a task without a predecessor task, assuming that task1 runs first, and when all the task slices of task1 run, the central node will then schedule task2 to run. [...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/46
CPCG06F9/4881G06F9/5038G06F9/5066G06F9/52
Inventor 才华
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap