Work scheduling method and work processor

A job scheduling and processor technology, applied in the communication field, can solve the problems of small proportion of data distribution nodes, head queue scheduling, and small input data size of small jobs, so as to reduce waiting time and ensure execution time

Active Publication Date: 2015-09-16
HANDAN BRANCH OF CHINA MOBILE GRP HEBEI COMPANYLIMITED
View PDF4 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Although the fair scheduling algorithm can ensure that all users get computing resources fairly, the execution of a large number of small tasks can easily lead to head queue scheduling problems: the input data of small jobs is small, and the nodes whose data is distributed in large-scale clusters account for the proportion of the entire cluster smaller
This causes the job to always run tasks on the same slot, which cannot guarantee better data locality and affects the throughput of the system.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Work scheduling method and work processor
  • Work scheduling method and work processor
  • Work scheduling method and work processor

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0032] The job scheduling method proposed by the embodiment of the present invention, such as figure 1 shown, including:

[0033] Step 101: When a new job is received, determine the latest execution time of the new job according to the arrival time and priority of the new job, and add the new job to the job according to the latest execution time cache queue.

[0034] Here, the priority may be that the client (Client) acquires the priority of the job to be uploaded according to a preset priority setting list; wherein, the priority setting list may be preset according to actual conditions.

[0035] This step can be completed by the job tracker (Job Tracker) in the prior art. For example: when the job tracker (Job Tracker) receives the i-th arriving job, it records the arrival time of the i-th job; according to the i-th The arrival time and priority of each job, and calculate the latest execution time of the i-th job.

[0036] The determining the latest execution time of the n...

Embodiment 2

[0056] The job processor that implements the method for job scheduling in Embodiment 1 provided by the embodiment of the present invention is as follows: figure 2 Shown include: job cache unit, queue organization unit, job execution unit; where,

[0057] The job buffering unit is configured to, when receiving a new job, determine the latest execution time of the new job according to the arrival time and priority of the new job, and store the new job according to the latest execution time The jobs are added to the job cache queue;

[0058] The queue organization unit is configured to determine the actual execution time of each job according to the latest execution time of each job in the job cache queue, and establish a job execution queue according to the actual execution time of each job in the job cache queue;

[0059] The job execution unit is configured to assign a task processor to the tasks obtained by decomposing each job in the job execution queue.

[0060] The prio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a work scheduling method and a work processor. The method comprises the steps that when new work is received, the latest executing time of the new work is determined according to the arrival time and the priority of the new work, and the new work is added to a work cache queue according to the latest executing time; the actual executing time of all pieces of work is determined according to the latest executing time of all the pieces of work in the work cache queue; according to the actual executing time of all the pieces of work in the work cache queue, a work executing queue is established, and task processors are allocated for tasks obtained by decomposing all the pieces of work in the work executing queue. By means of the work scheduling method and the work processor, the response time of the work with the high priority can be ensured, and the waiting time of a user is shortened.

Description

technical field [0001] The invention relates to scheduling technology in the communication field, in particular to a job scheduling method and a job processor. Background technique [0002] As a typical application framework of cloud computing, Map Reduce technology has been widely used by most large IT companies. Map Reduce simplifies the parallel software programming mode on super-large clusters composed of ordinary computers, and can be used for parallel processing of large-scale data sets. calculate. At present, there are mainly the following methods for job scheduling in the Map Reduce distributed computing system: first-in-first-out (FIFO, First Input First Output) method, HOD method, and fair scheduling algorithm. Wherein, the fair scheduling algorithm includes a fair sharing method and a minimum sharing quota method; the fair sharing method is: when a single job is running, it will use the entire cluster; when a new user submits a job, the system will Slots (task s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48G06F9/50
Inventor 李秀清苏彦志高梦娇
Owner HANDAN BRANCH OF CHINA MOBILE GRP HEBEI COMPANYLIMITED
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap