YARN-based GPGPU cluster-oriented resource management scheduling method

A technology of resource management and scheduling method, which is applied in the field of resource management and scheduling for GPGPU clusters, and can solve problems such as the inability to support GPU resource management and allocation.

Inactive Publication Date: 2017-12-08
BEIJING DIANZAN TECH CO LTD
View PDF2 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0011] The current YARN system supports the management and allocation of common resources such as CPU and memory, but cannot support the management and allocation of GPU resources.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • YARN-based GPGPU cluster-oriented resource management scheduling method
  • YARN-based GPGPU cluster-oriented resource management scheduling method
  • YARN-based GPGPU cluster-oriented resource management scheduling method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0045] Such as figure 2 As shown, the GPU cluster of the present invention is based on the improvement of the original YARN framework, and a GPU cluster management method is invented, so that GPU resources are visible, manageable, and schedulable at the cluster layer.

[0046] The YARN system of the present invention supports CPU (X86, ARM etc.), memory and GPU (the GPU of NVIDIA and the corresponding CUDA programming framework are used as examples in this valve, but the present invention is not limited to the GPU of NVIDIA).

[0047] The CPU and memory are "yarn.nodemanager.resource.memory-mb" and "yarn.nodemanager.resource.cpu-vcores", which are consistent with the original YARN, but need to increase the representation of GPU resources.

[0048] Add the representation of GPU type and quantity in YARN, for example, "yarn.nodemanager.resource.gpu-type" indicates the type of GPU, and "yarn.nodemanager.reosurce.gpu-ngpgpu" indicates the number of physically available general-purp...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a YARN-based GPGPU cluster-oriented resource management scheduling method. The method comprises the steps that a node manager reports node information to a resource manager through periodic heartbeats; the resource manager makes a response and triggers a NODE_UPDATE event of a scheduler; the scheduler allocates containers on nodes according to a scheduling policy and adds the containers to a resource allocation list; a GPU application manager sends heartbeats to the resource manager; the resource manager updates a resource application list and takes the containers of the resource application list as a response to the GPU application manager; the GPU application manager obtains the containers and performs second-layer scheduling; an application manager of resources except GPUs sends heartbeats to the resource manager; and the resource manager updates the resource application list and takes the containers of the resources except the GPUs in the list as a response to the application manager of the resources except GPUs. A YARN resource model is improved; and unified scheduling management of the GPUs by a cluster is realized by using the GPU application manager.

Description

technical field [0001] The invention belongs to the technical field of resource management of computer clusters, and in particular relates to a YARN-based resource management and scheduling method for GPGPU clusters. Background technique [0002] The full name of GPU in English is Graphic Processing Unit, and the Chinese translation is "graphics processing unit". GPU is a concept relative to CPU. Since the processing of graphics in modern computers (especially home systems and game enthusiasts) is becoming more and more important, a dedicated graphics core processor is required. GPU is the "heart" of the display card, which is equivalent to the role of the CPU in the computer. It determines the grade and most of the performance of the graphics card. Most of the graphics cards on the market now use graphics processing chips from NVIDIA and ATI. [0003] Today, GPU is no longer limited to 3D graphics processing. The development of GPU general-purpose computing technology has ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50
CPCG06F9/5027G06F2209/5013G06F2209/504
Inventor 张京梅
Owner BEIJING DIANZAN TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap