Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Pull mode and Push mode combined resource management and job scheduling method and system and medium

A resource management and job scheduling technology, which is applied in the fields of resource management and job scheduling, resource management and job scheduling of large-scale computer systems, and can solve the problems that affect the operating efficiency and scalability of the system, the performance bottleneck of the central scheduler system, and the system resources. Heterogeneity enhancement and other issues to achieve the effect of improving resource utilization, reducing bottleneck effect, and promoting scalability

Active Publication Date: 2020-08-07
SUN YAT SEN UNIV
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The so-called centralized means that the maintenance of job load information, the management of job queues, the maintenance of system resource information, and the job scheduling and allocation to idle resources are all undertaken by a specific central server or daemon process. There is a serious hidden danger in the architecture: the central scheduler can easily become the performance bottleneck and single point of failure of the entire system
Especially when the scale of the system expands, the type of workload becomes more and more complex, and the heterogeneity of system resources is further enhanced. If the centralized architecture is continued to be used and only the master node is used to complete all scheduling tasks, it will seriously affect the performance of the entire system. Operational Efficiency and Scalability

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Pull mode and Push mode combined resource management and job scheduling method and system and medium
  • Pull mode and Push mode combined resource management and job scheduling method and system and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] Such as figure 1 As shown, the implementation steps of the resource management and job scheduling method combining the Pull mode and the Push mode in this embodiment include:

[0044] 1) Receive jobs;

[0045] 2) Analyze or identify the job, and determine the type of job as high-performance computing job or big data processing job; it should be noted that different types of jobs can be stored in a mixed or separate manner according to needs, and the storage method can also be stored according to needs Use the required storage form, such as queue, linked list, etc.;

[0046] 3) Scheduling for different types of jobs, and for the high-performance computing jobs obtained by scheduling, use the Push mode to distribute: assign computing nodes to the high-performance computing jobs, and push the high-performance computing jobs to the assigned computing nodes for execution ; For the scheduled big data processing jobs, use the Pull mode to distribute: wait for the job request...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a Pull mode and Push mode combined resource management and job scheduling method and system and a medium. The method comprises the following steps: analyzing or identifying jobs to determine types and scheduling the jobs according to the types, and for high-performance computing jobs, distributing the jobs by adopting a Push mode: distributing computing nodes for the high-performance computing jobs, and pushing the high-performance computing jobs to the distributed computing nodes for execution; for the big data processing operation, adopting a Pull mode to perform distribution: waiting for an operation request of the computing node, and sending the big data processing operation to the computing node which actively sends the operation request to execute when the computing node actively sends the operation request. According to the method, the workload of the main node in the traditional system can be dispersed, the bottleneck effect caused by the workload is reduced, the resource utilization rate of the system is improved, and the method has the advantages of good universality, high resource utilization rate, high system throughput rate and good expandability.

Description

technical field [0001] The invention relates to the field of resource management and job scheduling of a large-scale computer system, in particular to a resource management and job scheduling method, system and medium combining a Pull mode and a Push mode. Background technique [0002] Resource management and job scheduling have always been a challenging problem in large-scale computing systems. At present, the scale of computing systems is getting larger and larger. For example, the Sunway TaihuLight supercomputer is composed of 40,000 computing nodes, and the number of processor cores reaches tens of millions. Shows heterogeneity, for example, the fastest Summit supercomputer currently contains 6 GPUs per node; since large-scale computing has become the basic means to promote technological progress in various industries, a large number of jobs with different characteristics are submitted to the computing system, lead to disorderly competition for resources. In the above ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/48G06F9/50
CPCG06F9/4881G06F9/5016G06F9/5027G06F2209/501G06F9/505
Inventor 陈志广卢宇彤肖侬
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com